Reality vs virtual reality racing

ABSTRACT

A method for displaying a virtual vehicle includes: calculating a virtual world comprising the virtual vehicle and a representation of a physical object at a virtual position; calculating a virtual position of a point of view within the virtual world based on a position of the point of view at the racecourse; and calculating a portion of the virtual vehicle within the virtual world that is visible from the virtual position of the point of view, wherein the portion of the virtual vehicle visible from the virtual position of the point of view comprises a portion of the virtual vehicle that is unobscured, from the virtual position of the point of view, by the representation of the physical object at the virtual position of the physical object.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/517,919, filed on Jul. 22, 2019 and which issued as U.S. Pat. No.10,953,330 on Mar. 23, 2021, which is a continuation of U.S. patentapplication Ser. No. 15/813,662, filed on Nov. 15, 2017 and which issuedas U.S. Pat. No. 10,357,715 on Jul. 23, 2019, which claims the benefitof U.S. Provisional Application No. 62/530,037, filed on Jul. 7, 2017,entitled “Racing Simulation,” which is herein incorporated by referencein its entirety.

FIELD

The present disclosure relates generally to vehicle simulation, and morespecifically to combining real world and virtual world automobileracing.

BACKGROUND

For almost every automobile, there is a racing competition. Our media isfull of car races, bike races, and truck races on racecourses. Each raceproduces a champion and, racing series produce season champions.

Automobile racing is not limited to the real world. Recently, virtualautomobile racing has gained popularity. Virtual racing champions canwin hundreds of thousands of dollars per race. Season virtual championscan win millions of dollars.

SUMMARY

In some embodiments, a method for displaying a virtual vehicle includesidentifying a position of a physical vehicle at a racecourse,identifying a position of a point of view at the racecourse, andproviding, to a display system, a portion of the virtual vehicle visiblefrom a virtual position of the point of view calculated within a virtualworld based on the position of the point of view at the racecourse. Asan exemplary advantage, embodiments described herein may allow aphysical vehicle operator to compete with a virtual vehicle operator.Further, by displaying a representation of the portion of the virtualvehicle visible from the virtual position of the point of view, thecompetition between the physical and virtual vehicle operators mayappear more realistic.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view includes a portion of the virtualvehicle that is unobscured, from the virtual position of the point ofview, by a representation of the physical vehicle at a virtual positionof the physical vehicle in the virtual world.

In some embodiments, the method further includes simulating, by thesimulation system, an interaction between the virtual vehicle and therepresentation of the physical vehicle in the virtual world, wherein theportion of the virtual vehicle visible from the virtual position of thepoint of view is calculated based on the interaction.

In some embodiments, the position of the point of view at the racecourseincludes a point of view of an operator of the physical vehicle, andidentifying a position of a point of view at the racecourse includesdetecting, at a sensor, the point of view of the operator of thephysical vehicle, the method further including: identifying a positionof a physical object; receiving kinematics information of the virtualvehicle; generating, at a display system, a representation of thevirtual vehicle based on the position of the physical object, theposition of the point of view at the racecourse, and the kinematicsinformation; and displaying the representation of the virtual vehiclesuch that the virtual vehicle is aligned with the physical object fromthe perspective of the position of the point of view at the racecourse.

In some embodiments, the method further includes generating, at adisplay system, the representation of the portion of the virtual vehiclevisible from the virtual position of the point of view.

In some embodiments, the method further includes displaying, by thedisplay system, a series of representations of the virtual vehicle overa period of time to simulate a trajectory of the virtual vehicle on theracecourse, the series of representations includes the representation ofthe portion of the virtual vehicle visible form the virtual position ofthe point of view. In some embodiments, a predicted trajectory of thevirtual vehicle is displayed. The prediction may be based on currenttrajectory, acceleration, current vehicle parameters, etc. This mayallow an audience member to anticipate if a virtual vehicle is likely toovertake a physical vehicle. The predicted trajectory may be presentedas a line, such as a yellow line. Other displays may also be included,such as “GOING TO PASS!” or “GOING TO CRASH!”

In some embodiments, the method further includes storing, by the displaysystem, a digital 3-D model of the virtual vehicle used to generate eachrepresentation from the series of representations, each representationis generated by the display system based on the digital 3-D model.

In some embodiments, the method further includes receiving a digital 3-Dmodel of the virtual vehicle used to generate each representation fromthe series of representations, each representation is generated by thedisplay system based on the digital 3-D model.

In some embodiments, the kinematics information includes one or morevectors of motion, one or more scalars of motion, a position vector, aGPS location, a velocity, an acceleration, an orientation, or acombination thereof of the virtual vehicle.

In some embodiments, identifying the position of the physical vehicleincludes detecting one or more vectors of motion, one or more scalars ofmotion, a position vector, a GPS location, a velocity, an acceleration,an orientation, or a combination thereof of the virtual vehicle.

In some embodiments, identifying the position of the point of view atthe racecourse includes detecting a spatial position of a head of anoperator of the physical vehicle.

In some embodiments, the method further includes transmitting, by atelemetry system coupled to the physical vehicle, the spatial positionto a simulator system; receiving, at the telemetry system, informationrelated to the portion of the virtual vehicle visible from the virtualposition of the point of view; and displaying, to the operator of thephysical vehicle, the representation of the portion of the virtualvehicle based on the information.

In some embodiments, the method further includes displaying therepresentation of the portion of the virtual vehicle includes:translating the information into a set of graphical elements, displayingthe representation of the portion includes displaying the set ofgraphical elements. In some embodiments, the method further includescomputing, at the simulation system, the information related to theportion visible from the virtual position of the point of view.

In some embodiments, displaying the series of representations of thevirtual vehicle includes displaying the series of representation on adisplay of the physical vehicle, and the display is a transparentorganic light-emitting diode (T-OLED) display that allows light to passthrough the T-OLED to display the field of view to the operator.

In some embodiments, displaying the series of representations of thevirtual vehicle includes displaying the series of representations on adisplay of the physical vehicle, and the display is an LCD display, themethod further including: capturing, by a camera coupled to the physicalvehicle, an image representing the field of view of the physical worldas seen by the operator on the display in the physical vehicle; andoutputting the image on a side of the LCD display to display the fieldof view to the operator, the series of representations are overlaid onthe image displayed by the LCD display.

In some embodiments, displaying the series of representations of thevirtual vehicle includes displaying the series of representations on adisplay of the physical vehicle, and the display includes a frontwindshield of the physical vehicle, one or more side windows of thephysical vehicle, a rear windshield of the physical vehicle, one or moreside mirrors, a rearview mirror, or a combination thereof.

In some embodiments, displaying the series of representations of thevirtual vehicle includes displaying the series of representations on adisplay of a headset worn by the operator. In some embodiments, theheadset is a helmet.

In some embodiments, identifying the position of the point of view atthe racecourse includes detecting one or more of a spatial position of auser's eyes, a gaze direction of the user's eyes, or a focus point ofthe user's eyes.

In some embodiments, the method further includes: providing the positionof the physical vehicle and the position of the point of view at theracecourse to a simulation system; calculating, by the simulationsystem, a virtual world including the virtual vehicle and arepresentation of the physical vehicle; calculating, by the simulationsystem, a virtual position of the point of view within the virtual worldbased on the position of the point of view at the racecourse; andcalculating, by the simulation system, the portion of the virtualvehicle visible from the virtual position of the point of view, andproviding, to a display system, the portion of the virtual vehiclevisible from the virtual position of the point of view includesoutputting, by the simulation system, the portion of the virtual vehiclevisible from the virtual position of the point of view.

In some embodiments, identifying the position of the physical vehicleincludes receiving a location of each of two portions of the vehicle.

In some embodiments, identifying the position of the physical vehicleincludes receiving a location of one portion of the vehicle and anorientation of the vehicle. In some embodiments, receiving theorientation of the vehicle includes receiving gyroscope data.

In some embodiments, the position of the point of view at the racecourseincludes a position of a point of view of an operator of the physicalvehicle at the racecourse. In some embodiments, the position of thepoint of view at the racecourse includes a position of a point of viewof an audience member present at a racecourse and observing the physicalvehicle on the racecourse.

In some embodiments, the position of the point of view at the racecourseincludes a position of a camera present at a racecourse and imaging thephysical vehicle on the racecourse. In some embodiments, the cameraimages a portion of the racecourse on which the physical vehicle isracing. When the physical vehicle is travelling across the portion ofthe racecourse being captured by the camera, the camera may capture thephysical vehicle in its video feed. When the physical vehicle is nottravelling across the portion of the racecourse being captured by thecamera, the camera may still capture the portion of the racecourse.

In some embodiments, identifying the position of the point of view atthe racecourse includes at least one of measuring a point of gaze ofeyes, tracking eye movement, tracking head position, identifying avector from one or both eyes to a fixed point on the physical vehicle,identifying a vector from a point on the head to a fixed point on thephysical vehicle, identifying a vector from a point on eye-wear to afixed point on the physical vehicle, identifying a vector from a pointon a head gear to a fixed point on the physical vehicle, identifying avector from one or both eyes to a fixed point in a venue, identifying avector from a point on the head to a fixed point in the venue,identifying a vector from a point on eye-wear to a fixed point in thevenue, or identifying a vector from a point on a head gear to a fixedpoint in the venue. In some embodiments, identifying the position of thepoint of view at the racecourse includes measuring the point of gaze ofthe eyes and the measuring includes measuring light reflection orrefraction from the eyes.

In some embodiments, providing the position of the physical vehicle andthe position of the point of view at the racecourse includes wirelesstransmitting at least one position.

In some embodiments, calculating a virtual world includes transformingphysical coordinates of the physical vehicle to coordinates in thevirtual world and the virtual position of the physical vehicle includesthe virtual coordinates.

In some embodiments, calculating the portion of the virtual vehiclevisible from the virtual position of the point of view includes:calculating a representation of the physical vehicle in the virtualworld, calculating a representation of a physical object in the virtualworld between the point of view and the virtual vehicle within thevirtual world, and extracting a portion of the virtual vehicle that isunobscured, from the virtual position of the point of view, by therepresentation of the physical vehicle and the representation of thephysical object. In some embodiments, the portion of the virtual vehiclewithin the virtual world that is visible from the virtual position ofthe point of view includes the unobscured portion.

In some embodiments, extracting the portions of the virtual vehicle mayinclude determining which pixels are obstructed by otherrepresentations, and only displaying pixels that are not obstructed byother representations. In some embodiments, extracting the portions ofthe virtual vehicle may include setting a pixel alpha value of zeropercent (in RGBA space) for all pixels obstructed by otherrepresentations. For example, portions of the virtual vehicle may beobstructed by other virtual representations, e.g., another virtualvehicle, or representations of physical objects, e.g., objects within aphysical vehicle or the physical vehicle itself. Any observed (from thevirtual position of the point of view) pixel values can be used toprovide the portions of the virtual vehicle that are visible from thevirtual position of the point of view. In some embodiments, the pixelsof unobscured and observed portions of the virtual vehicle can each beset to include an alpha value greater than zero percent (in RGBA space)to indicate that those unobscured pixels can be seen and should bedisplayed. In contrast, pixels set to an alpha value of zero percentindicate that those pixels are fully transparent, i.e., invisible, andwould not be displayed.

In some embodiments, calculating the representation of the physicalobject between the virtual position of the point of view and therepresentation of the physical vehicle includes accessing a database ofrepresentations to obtain a virtual position of the physical object.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view consists of portions of thevirtual vehicle that are unobscured by other representations in thevirtual world.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view includes a virtual shadow in thevirtual world. In some embodiments, the virtual shadow is at least oneof a shadow projected by the virtual vehicle and a shadow projected ontothe virtual vehicle. In some embodiments, the portion of the virtualvehicle visible from the virtual position of the point of view includesa virtual reflection. In some embodiments, the virtual reflection is atleast one of a reflection of the virtual vehicle and a reflection on thevirtual vehicle.

In some embodiments, calculating, by the simulation system, a portion ofthe virtual vehicle within the virtual world that is visible from thevirtual position of the point of view includes calculating a field ofview from the virtual position of the point of view and providing, tothe display system, the portion of the virtual vehicle visible from thevirtual position of the point of view includes displaying the portion ofthe virtual vehicle within the field of view.

In some embodiments, calculating, by the simulation system, a portion ofthe virtual vehicle within the virtual world that is visible from theposition of the virtual point of view includes calculating a field ofview from the virtual position of the point of view and providing, tothe display system, the portion of the virtual vehicle visible from thevirtual position of the point of view consists of displaying the portionof the virtual vehicle visible within the field of view.

In some embodiments, a method for displaying a virtual vehicle includesmeans for identifying a position of a physical vehicle at a racecourse,means for identifying a position of a point of view at the racecourse,and means for providing, to a display system, a portion of the virtualvehicle visible from a virtual position of the point of view calculatedwithin a virtual world based on the position of the point of view at theracecourse. As an exemplary advantage, embodiments described herein mayallow a physical vehicle operator to compete with a virtual vehicleoperator. Further, by displaying a representation of the portion of thevirtual vehicle visible from the virtual position of the point of view,the competition between the physical and virtual vehicle operators mayappear more realistic.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view includes a portion of the virtualvehicle that is unobscured, from the virtual position of the point ofview, by a representation of the physical vehicle at a virtual positionof the physical vehicle in the virtual world. In some embodiments, themethod further includes means for simulating, by the simulation system,an interaction between the virtual vehicle and the representation of thephysical vehicle in the virtual world, the portion of the virtualvehicle visible from the virtual position of the point of view iscalculated based on the interaction.

In some embodiments, the position of the point of view at the racecourseincludes a point of view of an operator of the physical vehicle, andmeans for identifying a position of a point of view at the racecourseincludes means for detecting, at a sensor, the point of view of theoperator of the physical vehicle, the method further including: meansfor identifying a position of a physical object; means for receivingkinematics information of the virtual vehicle; means for generating, ata display system, a representation of the virtual vehicle based on theposition of the physical object, the position of the point of view atthe racecourse, and the kinematics information; and means for displayingthe representation of the virtual vehicle such that the virtual vehicleis aligned with the physical object from the perspective of the positionof the point of view at the racecourse.

In some embodiments, the method further includes means for generating,at a display system, the representation of the portion of the virtualvehicle visible from the virtual position of the point of view.

In some embodiments, the method further includes means for displaying,by the display system, a series of representations of the virtualvehicle over a period of time to simulate a trajectory of the virtualvehicle on the racecourse, the series of representations includes therepresentation of the portion of the virtual vehicle visible form thevirtual position of the point of view. In some embodiments, a predictedtrajectory of the virtual vehicle is displayed. The prediction may bebased on current trajectory, acceleration, current vehicle parameters,etc. This may allow an audience member to anticipate if a virtualvehicle is likely to overtake a physical vehicle. The predictedtrajectory may be presented as a line, such as a yellow line. Otherdisplays may also be included, such as “GOING TO PASS!” or “GOING TOCRASH!”

In some embodiments, the method further includes means for storing, bythe display system, a digital 3-D model of the virtual vehicle used togenerate each representation from the series of representations, eachrepresentation is generated by the display system based on the digital3-D model.

In some embodiments, the method further includes means for receiving adigital 3-D model of the virtual vehicle used to generate eachrepresentation from the series of representations, each representationis generated by the display system based on the digital 3-D model.

In some embodiments, the kinematics information includes one or morevectors of motion, one or more scalars of motion, a position vector, aGPS location, a velocity, an acceleration, an orientation, or acombination thereof of the virtual vehicle.

In some embodiments, means for identifying the position of the physicalvehicle includes means for detecting one or more vectors of motion, oneor more scalars of motion, a position vector, a GPS location, avelocity, an acceleration, an orientation, or a combination thereof ofthe virtual vehicle.

In some embodiments, means for identifying the position of the point ofview at the racecourse includes means for detecting a spatial positionof a head of an operator of the physical vehicle. In some embodiments,the method further includes means for transmitting, by a telemetrysystem coupled to the physical vehicle, the spatial position to asimulator system; means for receiving, at the telemetry system,information related to the portion of the virtual vehicle visible fromthe virtual position of the point of view; and means for displaying, tothe operator of the physical vehicle, the representation of the portionof the virtual vehicle based on the information.

In some embodiments, the method further includes means for displayingthe representation of the portion of the virtual vehicle includes: meansfor translating the information into a set of graphical elements, meansfor displaying the representation of the portion includes means fordisplaying the set of graphical elements. In some embodiments, themethod further includes means for computing, at the simulation system,the information related to the portion visible from the virtual positionof the point of view.

In some embodiments, means for displaying the series of representationsof the virtual vehicle includes means for displaying the series ofrepresentation on a display of the physical vehicle, and the display isa transparent organic light-emitting diode (T-OLED) display that allowslight to pass through the T-OLED to display the field of view to theoperator.

In some embodiments, means for displaying the series of representationsof the virtual vehicle includes means for displaying the series ofrepresentations on a display of the physical vehicle, and the display isan LCD display, the method further including: means for capturing, by acamera coupled to the physical vehicle, an image representing the fieldof view of the physical world as seen by the operator on the display inthe physical vehicle; and means for outputting the image on a side ofthe LCD display to display the field of view to the operator, the seriesof representations are overlaid on the image displayed by the LCDdisplay.

In some embodiments, means for displaying the series of representationsof the virtual vehicle includes means for displaying the series ofrepresentations on a display of the physical vehicle, and the displayincludes a front windshield of the physical vehicle, one or more sidewindows of the physical vehicle, a rear windshield of the physicalvehicle, one or more side mirrors, a rearview mirror, or a combinationthereof.

In some embodiments, means for displaying the series of representationsof the virtual vehicle includes means for displaying the series ofrepresentations on a display of a headset worn by the operator. In someembodiments, the headset is a helmet.

In some embodiments, means for identifying the position of the point ofview at the racecourse includes means for detecting one or more of aspatial position of a user's eyes, a gaze direction of the user's eyes,or a focus point of the user's eyes.

In some embodiments, the method further includes: means for providingthe position of the physical vehicle and the position of the point ofview at the racecourse to a simulation system; means for calculating, bythe simulation system, a virtual world including the virtual vehicle anda representation of the physical vehicle; means for calculating, by thesimulation system, a virtual position of the point of view within thevirtual world based on the position of the point of view at theracecourse; and means for calculating, by the simulation system, theportion of the virtual vehicle visible from the virtual position of thepoint of view, and means for providing, to a display system, the portionof the virtual vehicle visible from the virtual position of the point ofview includes means for outputting, by the simulation system, theportion of the virtual vehicle visible from the virtual position of thepoint of view.

In some embodiments, means for identifying the position of the physicalvehicle includes means for receiving a location of each of two portionsof the vehicle.

In some embodiments, means for identifying the position of the physicalvehicle includes means for receiving a location of one portion of thevehicle and an orientation of the vehicle. In some embodiments, meansfor receiving the orientation of the vehicle includes means forreceiving gyroscope data.

In some embodiments, the position of the point of view at the racecourseincludes a position of a point of view of an operator of the physicalvehicle at the racecourse. In some embodiments, the position of thepoint of view at the racecourse includes a position of a point of viewof an audience member present at a racecourse and observing the physicalvehicle on the racecourse. In some embodiments, the position of thepoint of view at the racecourse includes a position of a camera presentat a racecourse and the method further includes means for imaging thephysical vehicle on the racecourse. In some embodiments, the cameraimages a portion of the racecourse on which the physical vehicle isracing. When the physical vehicle is travelling across the portion ofthe racecourse being captured by the camera, the camera may capture thephysical vehicle in its video feed. When the physical vehicle is nottravelling across the portion of the racecourse being captured by thecamera, the camera may still capture the portion of the racecourse.

In some embodiments, means for identifying the position of the point ofview at the racecourse includes at least one of means for measuring apoint of gaze of eyes, means for tracking eye movement, means fortracking head position, means for identifying a vector from one or botheyes to a fixed point on the physical vehicle, means for identifying avector from a point on the head to a fixed point on the physicalvehicle, means for identifying a vector from a point on eye-wear to afixed point on the physical vehicle, means for identifying a vector froma point on a head gear to a fixed point on the physical vehicle, meansfor identifying a vector from one or both eyes to a fixed point in avenue, means for identifying a vector from a point on the head to afixed point in the venue, means for identifying a vector from a point oneye-wear to a fixed point in the venue, or means for identifying avector from a point on a head gear to a fixed point in the venue. Insome embodiments, means for identifying the position of the point ofview at the racecourse includes means for measuring the point of gaze ofthe eyes and the means for measuring includes means for measuring lightreflection or refraction from the eyes.

In some embodiments, means for providing the position of the physicalvehicle and the position of the point of view at the racecourse includesmeans for wireless transmitting at least one position.

In some embodiments, means for calculating a virtual world includesmeans for transforming physical coordinates of the physical vehicle tocoordinates in the virtual world and the virtual position of thephysical vehicle includes the virtual coordinates.

In some embodiments, means for calculating the portion of the virtualvehicle visible from the virtual position of the point of view includes:means for calculating a representation of the physical vehicle in thevirtual world, means for calculating a representation of a physicalobject in the virtual world between the point of view and the virtualvehicle within the virtual world, and means for extracting a portion ofthe virtual vehicle that is unobscured, from the virtual position of thepoint of view, by the representation of the physical vehicle and therepresentation of the physical object. In some embodiments, the portionof the virtual vehicle within the virtual world that is visible from thevirtual position of the point of view includes the unobscured portion.

In some embodiments, means for extracting the portions of the virtualvehicle may include means for determining which pixels are obstructed byother representations, and only displaying pixels that are notobstructed by other representations. In some embodiments, means forextracting the portions of the virtual vehicle may include means forsetting a pixel alpha value of zero percent (in RGBA space) for allpixels obstructed by other representations. For example, portions of thevirtual vehicle may be obstructed by other virtual representations,e.g., another virtual vehicle, or representations of physical objects,e.g., objects within a physical vehicle or the physical vehicle itself.Any observed (from the virtual position of the point of view) pixelvalues can be used to provide the portions of the virtual vehicle thatare visible from the virtual position of the point of view. In someembodiments, the pixels of unobscured and observed portions of thevirtual vehicle can each be set to include an alpha value greater thanzero percent (in RGBA space) to indicate that those unobscured pixelscan be seen and should be displayed. In contrast, pixels set to an alphavalue of zero percent indicate that those pixels are fully transparent,i.e., invisible, and would not be displayed.

In some embodiments, means for calculating the representation of thephysical object between the virtual position of the point of view andthe representation of the physical vehicle includes means for accessinga database of representations to obtain a virtual position of thephysical object.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view consists of portions of thevirtual vehicle that are unobscured by other representations in thevirtual world.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view includes a virtual shadow in thevirtual world. In some embodiments, the virtual shadow is at least oneof a shadow projected by the virtual vehicle and a shadow projected ontothe virtual vehicle. In some embodiments, the portion of the virtualvehicle visible from the virtual position of the point of view includesa virtual reflection. In some embodiments, the virtual reflection is atleast one of a reflection of the virtual vehicle and a reflection on thevirtual vehicle.

In some embodiments, means for calculating, by the simulation system, aportion of the virtual vehicle within the virtual world that is visiblefrom the virtual position of the point of view includes means forcalculating a field of view from the virtual position of the point ofview and means for providing, to the display system, the portion of thevirtual vehicle visible from the virtual position of the point of viewincludes displaying the portion of the virtual vehicle within the fieldof view.

In some embodiments, means for calculating, by the simulation system, aportion of the virtual vehicle within the virtual world that is visiblefrom the position of the virtual point of view includes means forcalculating a field of view from the virtual position of the point ofview and means for providing, to the display system, the portion of thevirtual vehicle visible from the virtual position of the point of viewconsists of means for displaying the portion of the virtual vehiclevisible within the field of view.

In some embodiments, a system for displaying virtual vehicles includes afirst sensor detecting a position of a physical vehicle at a racecourse,a second sensor detecting a position of a point of view at theracecourse, and a simulation system outputting a portion of the virtualvehicle visible from a virtual position of the point of view. As anexemplary advantage, embodiments described herein may allow a physicalvehicle operator to compete with a virtual vehicle operator. Further, bydisplaying a portion of the virtual vehicle visible from the position ofthe point of view at the racecourse, the competition between thephysical and virtual vehicle operators may appear more realistic.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view includes a portion of the virtualvehicle that is unobscured, from the virtual position of the point ofview, by a representation of the physical vehicle at a virtual positionof the physical vehicle in the virtual world.

In some embodiments, the system further includes a simulation systemconfigured to simulate an interaction between the virtual vehicle andthe representation of the physical vehicle in the virtual world, theportion of the virtual vehicle visible from the virtual position of thepoint of view is calculated based on the interaction.

In some embodiments, the system further includes the first sensor iscoupled to the physical vehicle, the second sensor is configured todetect eye position of an operator of the physical vehicle, and thedisplay system is coupled to the physical vehicle and configured to:receive kinematics information of a virtual vehicle; identify a positionof a physical object in the field of view of the operator, generate arepresentation of the virtual vehicle based on the position of thephysical object, the point of view, and the kinematics information; anddisplay the representation of the virtual vehicle such that the virtualvehicle is aligned with the physical object from the perspective of thepoint of view.

In some embodiments, the system further includes a display systemconfigured to generate the representation of the portion of the virtualvehicle visible from the virtual position of the point of view.

In some embodiments, displaying the representation of the virtualvehicle includes displaying the representation on a display screen ofthe physical vehicle, and the display system is further configured to:display, on the display screen, a series of representations of thevirtual vehicle over a period of time to simulate a trajectory of thevirtual vehicle on the racecourse, the series of representationsincludes the representation of the portion of the virtual vehiclevisible form the virtual position of the point of view. In someembodiments, a predicted trajectory of the virtual vehicle is bedisplayed. The prediction may be based on current trajectory,acceleration, current vehicle parameters, etc. This may allow anaudience member to anticipate if a virtual vehicle is likely to overtakea physical vehicle. The predicted trajectory may be presented as a line,such as a yellow line. Other displays may also be included, such as“GOING TO PASS!” or “GOING TO CRASH!”

In some embodiments, displaying the series of representations of thevirtual vehicle includes displaying the series of representations on adisplay of the physical vehicle, and the display system is furtherconfigured to: store a digital 3-D model of the virtual vehicle used togenerate each representation from the series of representations, eachrepresentation is generated by the display system based on the digital3-D model.

In some embodiments, the display system is further configured to receivea digital 3-D model of the virtual vehicle used to generate eachrepresentation from the series of representations, each representationis generated by the display system based on the digital 3-D model.

In some embodiments, the kinematics information includes one or morevectors of motion, one or more scalars of motion, a position vector, aGPS location, a velocity, an acceleration, an orientation, or acombination thereof of the virtual vehicle.

In some embodiments, the first sensor is configured to detect theposition of the physical vehicle by detecting one or more vectors ofmotion, one or more scalars of motion, a position vector, a GPSlocation, a velocity, an acceleration, an orientation, or a combinationthereof of the virtual vehicle.

In some embodiments, the first sensor is configured to detect the pointof view of the operator by: detecting a spatial position of a head ofthe operator.

In some embodiments, the display system is further configured to:transmit the spatial position to a simulation system; receiveinformation related to the portion of the virtual vehicle visible fromthe virtual position of the point of view; and display, to the operatorof the physical vehicle, the portion of the virtual vehicle.

In some embodiments, the display system is further configured to displaythe series of representations of the virtual vehicle by: translating theinformation into a set of graphical elements, displaying therepresentation of the portion includes displaying the set of graphicalelements. In some embodiments, the information related to the portionvisible from the virtual position of the point of view is computed atthe simulation system.

In some embodiments, the system further includes: a transparent organiclight-emitting diode (T-OLED) display that allows light to pass throughthe T-OLED to display the field of view to the operator, and the displaysystem is configured to display the series of representations on theT-OLED display to display the representation of the virtual vehicle.

In some embodiments, the system further includes an LCD display; acamera coupled to the physical vehicle and configured to capture animage representing the field of view of the physical world as seen bythe operator on the LCD display in the physical vehicle, and the displaysystem is configured to: output the image on a side of the LCD displayto display the field of view to the operator, and overlay the series ofrepresentations on the image displayed by the LCD display.

In some embodiments, the system further includes a display that includesa windshield of the physical vehicle, one or more side windows of thephysical vehicle, a rear windshield of the physical vehicle, or acombination thereof, and the display system is configured to display therepresentation of the virtual vehicle on the display.

In some embodiments, the system includes a headset worn by the operator,the headset including a display, and the display system is configured todisplay the series of representations of the virtual vehicle on thedisplay. In some embodiments, the headset is a helmet.

In some embodiments, the second sensor detects one or more of a spatialposition of a user's eyes, a gaze direction of the user's eyes, or afocus point of the user's eyes.

In some embodiments, the system includes a simulation system configuredto: receive the position of the physical vehicle and the position of thepoint of view at the racecourse; calculate a virtual world including thevirtual vehicle and a representation of the physical vehicle; calculatea virtual position of the point of view within the virtual world basedon the position of the point of view at the racecourse; calculate theportion of the virtual vehicle visible from the virtual position of thepoint of view; and output, to the display system, the portion of thevirtual vehicle visible from the virtual position of the point of view.

In some embodiments, the first sensor receives a location of each of twoportions of the vehicle.

In some embodiments, the first sensor receives a location of one portionof the vehicle and an orientation of the vehicle. In some embodiments,receiving the orientation of the vehicle includes receiving gyroscopedata.

In some embodiments, the position of the point of view at the racecourseincludes a position of a point of view of an operator of the physicalvehicle at the racecourse. In some embodiments, the position of thepoint of view at the racecourse includes a position of a point of viewof an audience member present at a racecourse and observing the physicalvehicle on the racecourse.

In some embodiments, the position of the point of view at the racecourseincludes a position of a camera present at a racecourse and imaging thephysical vehicle on the racecourse. In some embodiments, the cameraimages a portion of the racecourse on which the physical vehicle isracing. When the physical vehicle is travelling across the portion ofthe racecourse being captured by the camera, the camera may capture thephysical vehicle in its video feed. When the physical vehicle is nottravelling across the portion of the racecourse being captured by thecamera, the camera may still capture the portion of the racecourse.

In some embodiments, the second sensor is configured to detect theposition of the point of view at the racecourse by at least one ofmeasuring the point of gaze of eyes, tracking eye movement, trackinghead position, identifying a vector from one or both eyes to a fixedpoint on the physical vehicle, identifying a vector from a point on thehead to a fixed point on the physical vehicle, identifying a vector froma point on eye-wear to a fixed point on the physical vehicle,identifying a vector from a point on a head gear to a fixed point on thephysical vehicle, identifying a vector from one or both eyes to a fixedpoint in a venue, identifying a vector from a point on the head to afixed point in the venue, or identifying a vector from a point oneye-wear to a fixed point in the venue, identifying a vector from apoint on a head gear to a fixed point in the venue. In some embodiments,identifying the position of the point of view at the racecourse includesmeasuring the point of gaze of the eyes and the measuring includesmeasuring light reflection or refraction from the eyes.

In some embodiments, receiving the position of the physical vehicle andthe position of the point of view at the racecourse includes wirelessreceiving at least one position.

In some embodiments, calculating a virtual world includes transformingphysical coordinates of the physical vehicle to coordinates in thevirtual world and the virtual position of the physical vehicle includesthe virtual coordinates.

In some embodiments, calculating a portion of the virtual vehiclevisible from the virtual position of the point of view includes:calculating a representation of the physical vehicle in the virtualworld, calculating a representation of a physical object in the virtualworld between the point of view and the virtual vehicle within thevirtual world, and extracting a portion of the virtual vehicle that isunobscured, from the virtual position of the point of view, by therepresentation of the physical vehicle and the representation of thephysical object. In some embodiments, the portion of the virtual vehiclewithin the virtual world that is visible from the virtual position ofthe point of view includes the unobscured portion.

In some embodiments, extracting the portions of the virtual vehicle mayinclude determining which pixels are obstructed by otherrepresentations, and only displaying pixels that are not obstructed byother representations. In some embodiments, extracting the portions ofthe virtual vehicle may include setting a pixel alpha value of zeropercent (in RGBA space) for all pixels obstructed by otherrepresentations. For example, portions of the virtual vehicle may beobstructed by other virtual representations, e.g., another virtualvehicle, or representations of physical objects, e.g., objects within aphysical vehicle or the physical vehicle itself. Any observed (from thevirtual position of the point of view) pixel values can be used toprovide the portions of the virtual vehicle that are visible from thevirtual position of the point of view. In some embodiments, the pixelsof unobscured and observed portions of the virtual vehicle can each beset to include an alpha value greater than zero percent (in RGBA space)to indicate that those unobscured pixels can be seen and should bedisplayed. In contrast, pixels set to an alpha value of zero percentindicate that those pixels are fully transparent, i.e., invisible, andwould not be displayed.

In some embodiments, calculating the representation of the physicalobject between the virtual position of the point of view and therepresentation of the physical vehicle includes accessing a database ofrepresentations to obtain a virtual position of the physical object.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view consists of portions of thevirtual vehicle that are unobscured by other representations in thevirtual world.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view includes a virtual shadow in thevirtual world. In some embodiments, the virtual shadow is at least oneof a shadow projected by the virtual vehicle and a shadow projected ontothe virtual vehicle. In some embodiments, the portion of the virtualvehicle visible from the position of the point of view at the racecourseincludes a virtual reflection. In some embodiments, the virtualreflection is at least one of a reflection of the virtual vehicle and areflection on the virtual vehicle.

In some embodiments, the system further includes the simulation systemconfigured to calculate a portion of the virtual vehicle visible fromthe virtual position of the point of view by calculating a field of viewfrom the virtual position of the point of view and providing, to thedisplay system, the portion of the virtual vehicle visible from thevirtual position of the point of view includes displaying the portion ofthe virtual vehicle within the field of view.

In some embodiments, the system further includes the simulation systemconfigured to calculate a portion of the virtual vehicle visible fromthe virtual position of the point of view by calculating a field of viewfrom the virtual position of the point of view and providing, to thedisplay system, the portion of the virtual vehicle visible from thevirtual position of the point of view consists of displaying the portionof the virtual vehicle visible within the field of view.

DESCRIPTION OF THE FIGURES

FIG. 1 is a diagram of a physical vehicle, according to someembodiments.

FIGS. 2A-C are diagrams showing how one or more virtual vehicles aredisplayed on one or more displays, according to some embodiments.

FIGS. 3A-D are diagrams showing how visible portions of virtual vehiclesare displayed on a display, according to some embodiments.

FIGS. 4A-D are diagrams showing how visible portions of virtual vehiclesare displayed on a display, according to some embodiments.

FIG. 5 is a system for simulating a virtual race between a live systemand a simulated system, according to some embodiments.

FIG. 6 is a flowchart illustrating a method for displaying virtualvehicles on displays, according to some embodiments.

FIG. 7 is a flowchart illustrating a method for providing two-wayinteractions between a virtual vehicle and a physical vehicle to anoperator of the physical vehicle, according to some embodiments.

FIG. 8 is a flowchart illustrating a method for simulating a racebetween a virtual vehicle and a physical vehicle to provide two-wayinteractions, according to some embodiments.

FIG. 9 is a flowchart illustrating a method performed by a simulationsystem to enable display of virtual vehicles, according to someembodiments.

FIG. 10 is a flowchart illustrating a method to enable display ofvirtual vehicles, according to some embodiments.

FIG. 11 is a functional block diagram of a computer in accordance withsome embodiments.

DETAILED DESCRIPTION

Embodiments described herein merge real world and virtual world racingcompetitions. For example, real world racing champions and virtual worldracing champions can compete to determine an overall champion.Advantageously, each champion can stay within their respective “world”and still compete with a champion from another “world.” In effect,embodiments described herein enable live participants to compete againstvirtual participants.

The terms “physical” and “real-world” are used interchangeably hereinand to contrast with “virtual world.” For example, a “physical vehicle”or “real-world vehicle” can be physically present on or at a racecourse.A “virtual vehicle” cannot be physically present on the same racecourse.For example, a “virtual vehicle” may be a graphically generated vehiclethat is shown on a display. In some embodiments, a “virtual vehicle” isa representation in a software-based environment.

In some embodiments, a method for displaying a virtual vehicle includesidentifying a position of a physical vehicle at a racecourse,identifying a position of a point of view at the racecourse, andproviding, to a display system, a portion of the virtual vehicle visiblefrom a virtual position of the point of view. Problems solved byembodiments disclosed herein can include overcoming the lack of realismexperienced by users of prior solutions. In some embodiments herein,providing visible portions of the virtual vehicle to the user increasesthe realism experienced by the user. The increased realism provides areliable and re-producible user experience by providing a real-worldrace that includes a virtual vehicle.

In some embodiments, the visible portion of the virtual vehicle iscalculated based on a virtual position of the physical vehicle in avirtual world, a virtual position of the point of view in the virtualworld, and a virtual position of the virtual vehicle in the virtualworld. Problems solved by embodiments disclosed herein can include howto provide a visible portion of a virtual vehicle. In some embodimentsherein, providing visible portions of the virtual vehicle through avirtual calculation of the visible portion increases the accuracy of thevisible portion determination. The increased accuracy provides areliable and re-producible user experience by providing a real-worldrace that includes a virtual vehicle. In some embodiments herein,providing visible portions through a virtual calculation increases theefficiency of the calculation. The increased efficiency reduces powerusage and improves representation speed for a more seamless userexperience.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view includes a portion of the virtualvehicle that is unobscured, from the virtual position of the point ofview, by a representation of the physical vehicle at a virtual positionof the physical vehicle in the virtual world.

In some embodiments, the method further includes simulating, by thesimulation system, an interaction between the virtual vehicle and therepresentation of the physical vehicle in the virtual world, the portionof the virtual vehicle visible from the virtual position of the point ofview is calculated based on the interaction.

In some embodiments, the position of the point of view at the racecourseincludes a point of view of an operator of the physical vehicle, andidentifying a position of a point of view at the racecourse includesdetecting, at a sensor, the point of view of the operator of thephysical vehicle, the method further including: identifying a positionof a physical object; receiving kinematics information of the virtualvehicle; generating, at a display system, a representation of thevirtual vehicle based on the position of the physical object, theposition of the point of view at the racecourse, and the kinematicsinformation; and displaying the representation of the virtual vehiclesuch that the virtual vehicle is aligned with the physical object fromthe perspective of the position of the point of view at the racecourse.

In some embodiments, the method further includes generating, at adisplay system, the representation of the portion of the virtual vehiclevisible from the virtual position of the point of view.

In some embodiments, the method further includes displaying, by thedisplay system, a series of representations of the virtual vehicle overa period of time to simulate a trajectory of the virtual vehicle on theracecourse, the series of representations includes the representation ofthe portion of the virtual vehicle visible form the virtual position ofthe point of view. In some embodiments, a predicted trajectory of thevirtual vehicle is displayed. The prediction may be based on currenttrajectory, acceleration, current vehicle parameters, etc. This mayallow an audience member to anticipate if a virtual vehicle is likely toovertake a physical vehicle. The predicted trajectory may be presentedas a line, such as a yellow line. Other displays may also be included,such as “GOING TO PASS!” or “GOING TO CRASH!”

In some embodiments, the method further includes storing, by the displaysystem, a digital 3-D model of the virtual vehicle used to generate eachrepresentation from the series of representations, each representationis generated by the display system based on the digital 3-D model.

In some embodiments, the method further includes receiving a digital 3-Dmodel of the virtual vehicle used to generate each representation fromthe series of representations, each representation is generated by thedisplay system based on the digital 3-D model.

In some embodiments, the kinematics information includes one or morevectors of motion, one or more scalars of motion, a position vector, aGPS location, a velocity, an acceleration, an orientation, or acombination thereof of the virtual vehicle.

In some embodiments, identifying the position of the physical vehicleincludes detecting one or more vectors of motion, one or more scalars ofmotion, a position vector, a GPS location, a velocity, an acceleration,an orientation, or a combination thereof of the virtual vehicle.

In some embodiments, identifying the position of the point of view atthe racecourse includes detecting a spatial position of a head of anoperator of the physical vehicle. In some embodiments, the methodfurther includes transmitting, by a telemetry system coupled to thephysical vehicle, the spatial position to a simulator system; receiving,at the telemetry system, information related to the portion of thevirtual vehicle visible from the virtual position of the point of view;and displaying, to the operator of the physical vehicle, therepresentation of the portion of the virtual vehicle based on theinformation.

In some embodiments, the method further includes displaying therepresentation of the portion of the virtual vehicle includes:translating the information into a set of graphical elements, displayingthe representation of the portion includes displaying the set ofgraphical elements. In some embodiments, the method further includescomputing, at the simulation system, the information related to theportion visible from the virtual position of the point of view.

In some embodiments, displaying the series of representations of thevirtual vehicle includes displaying the series of representation on adisplay of the physical vehicle, and the display is a transparentorganic light-emitting diode (T-OLED) display that allows light to passthrough the T-OLED to display the field of view to the operator.

In some embodiments, displaying the series of representations of thevirtual vehicle includes displaying the series of representations on adisplay of the physical vehicle, and the display is an LCD display, themethod further including: capturing, by a camera coupled to the physicalvehicle, an image representing the field of view of the physical worldas seen by the operator on the display in the physical vehicle; andoutputting the image on a side of the LCD display to display the fieldof view to the operator, the series of representations are overlaid onthe image displayed by the LCD display.

In some embodiments, displaying the series of representations of thevirtual vehicle includes displaying the series of representations on adisplay of the physical vehicle, and the display includes a frontwindshield of the physical vehicle, one or more side windows of thephysical vehicle, a rear windshield of the physical vehicle, one or moreside mirrors, a rearview mirror, or a combination thereof.

In some embodiments, displaying the series of representations of thevirtual vehicle includes displaying the series of representations on adisplay of a headset worn by the operator. In some embodiments, theheadset is a helmet.

In some embodiments, identifying the position of the point of view atthe racecourse includes detecting one or more of a spatial position of auser's eyes, a gaze direction of the user's eyes, or a focus point ofthe user's eyes.

In some embodiments, the method further includes: providing the positionof the physical vehicle and the position of the point of view at theracecourse to a simulation system; calculating, by the simulationsystem, a virtual world including the virtual vehicle and arepresentation of the physical vehicle; calculating, by the simulationsystem, a virtual position of the point of view within the virtual worldbased on the position of the point of view at the racecourse; andcalculating, by the simulation system, the portion of the virtualvehicle visible from the virtual position of the point of view, andproviding, to a display system, the portion of the virtual vehiclevisible from the virtual position of the point of view includesoutputting, by the simulation system, the portion of the virtual vehiclevisible from the virtual position of the point of view. Problems solvedby embodiments disclosed herein can include how to calculate a visibleportion of a virtual vehicle. In some embodiments herein, calculatingthe visible portion of the virtual vehicle in a virtual world increasesthe accuracy of the visible portion determination. The increasedaccuracy provides a reliable and re-producible user experience byproviding a real-world race that includes a virtual vehicle. In someembodiments herein, providing visible portions through a virtualcalculation increases the efficiency of the calculation. The increasedefficiency reduces power usage and improves representation speed for amore seamless user experience.

In some embodiments, identifying the position of the physical vehicleincludes receiving a location of each of two portions of the vehicle. Insome embodiments, identifying the position of the physical vehicleincludes receiving a location of one portion of the vehicle and anorientation of the vehicle. In some embodiments, receiving theorientation of the vehicle includes receiving gyroscope data. Problemssolved by embodiments disclosed herein can include how to correctlyposition a physical vehicle in a virtual world for determining a visibleportion of a virtual vehicle. In some embodiments herein, using ameasure of orientation provides for accurate placement of the physicalvehicle in the virtual world. The increased accuracy provides for a morefaithful display of the visible portions of the vehicle, therebyimproving the user experience.

In some embodiments, the position of the point of view at the racecourseincludes a position of a point of view of an operator of the physicalvehicle at the racecourse. In some embodiments, the position of thepoint of view at the racecourse includes a position of a point of viewof an audience member present at a racecourse and observing the physicalvehicle on the racecourse. In some embodiments, the position of thepoint of view at the racecourse includes a position of a camera presentat a racecourse and imaging the physical vehicle on the racecourse. Insome embodiments, the camera images a portion of the racecourse on whichthe physical vehicle is racing. When the physical vehicle is travellingacross the portion of the racecourse being captured by the camera, thecamera may capture the physical vehicle in its video feed. When thephysical vehicle is not travelling across the portion of the racecoursebeing captured by the camera, the camera may still capture the portionof the racecourse.

In some embodiments, identifying the position of the point of view atthe racecourse includes at least one of measuring a point of gaze ofeyes, tracking eye movement, tracking head position, identifying avector from one or both eyes to a fixed point on the physical vehicle,identifying a vector from a point on the head to a fixed point on thephysical vehicle, identifying a vector from a point on eye-wear to afixed point on the physical vehicle, identifying a vector from a pointon a head gear to a fixed point on the physical vehicle, identifying avector from one or both eyes to a fixed point in a venue, identifying avector from a point on the head to a fixed point in the venue,identifying a vector from a point on eye-wear to a fixed point in thevenue, or identifying a vector from a point on a head gear to a fixedpoint in the venue. In some embodiments, identifying the position of thepoint of view at the racecourse includes measuring the point of gaze ofthe eyes and the measuring includes measuring light reflection orrefraction from the eyes.

In some embodiments, providing the position of the physical vehicle andthe position of the point of view at the racecourse includes wirelesstransmitting at least one position.

In some embodiments, calculating a virtual world includes transformingphysical coordinates of the physical vehicle to coordinates in thevirtual world and the virtual position of the physical vehicle includesthe virtual coordinates.

In some embodiments, calculating the portion of the virtual vehiclevisible from the virtual position of the point of view includes:calculating a representation of the physical vehicle in the virtualworld, calculating a representation of a physical object in the virtualworld between the point of view and the virtual vehicle within thevirtual world, and extracting a portion of the virtual vehicle that isunobscured, from the virtual position of the point of view, by therepresentation of the physical vehicle and the representation of thephysical object. In some embodiments, the portion of the virtual vehiclewithin the virtual world that is visible from the virtual position ofthe point of view includes the unobscured portion. Problems solved byembodiments disclosed herein can include how to calculate a visibleportion of a virtual vehicle, including more than just the portion thatis not obscured by the physical vehicle. In some embodiments herein,calculating the visible portion in a virtual world that includesphysical objects in the real world increases the accuracy of the visibleportion determination. The increased accuracy provides a reliable andre-producible user experience by providing a real-world race thatincludes a virtual vehicle. In some embodiments herein, providingvisible portions through a virtual calculation increases the efficiencyof the calculation. The increased efficiency reduces power usage andimproves representation speed for a more seamless user experience.

In some embodiments, extracting the portions of the virtual vehicle mayinclude determining which pixels are obstructed by otherrepresentations, and only displaying pixels that are not obstructed byother representations. In some embodiments, extracting the portions ofthe virtual vehicle may include setting a pixel alpha value of zeropercent (in RGBA space) for all pixels obstructed by otherrepresentations. For example, portions of the virtual vehicle may beobstructed by other virtual representations, e.g., another virtualvehicle, or representations of physical objects, e.g., objects within aphysical vehicle or the physical vehicle itself. Any observed (from thevirtual position of the point of view) pixel values can be used toprovide the portions of the virtual vehicle that are visible from thevirtual position of the point of view. In some embodiments, the pixelsof unobscured and observed portions of the virtual vehicle can each beset to include an alpha value greater than zero percent (in RGBA space)to indicate that those unobscured pixels can be seen and should bedisplayed. In contrast, pixels set to an alpha value of zero percentindicate that those pixels are fully transparent, i.e., invisible, andwould not be displayed.

In some embodiments, calculating the representation of the physicalobject between the virtual position of the point of view and therepresentation of the physical vehicle includes accessing a database ofrepresentations to obtain a virtual position of the physical object.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view consists of portions of thevirtual vehicle that are unobscured by other representations in thevirtual world.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view includes a virtual shadow in thevirtual world. In some embodiments, the virtual shadow is at least oneof a shadow projected by the virtual vehicle and a shadow projected ontothe virtual vehicle. In some embodiments, the portion of the virtualvehicle visible from the virtual position of the point of view includesa virtual reflection. In some embodiments, the virtual reflection is atleast one of a reflection of the virtual vehicle and a reflection on thevirtual vehicle.

In some embodiments, calculating, by the simulation system, a portion ofthe virtual vehicle within the virtual world that is visible from thevirtual position of the point of view includes calculating a field ofview from the virtual position of the point of view and providing, tothe display system, the portion of the virtual vehicle visible from thevirtual position of the point of view includes displaying the portion ofthe virtual vehicle within the field of view.

In some embodiments, calculating, by the simulation system, a portion ofthe virtual vehicle within the virtual world that is visible from theposition of the virtual point of view includes calculating a field ofview from the virtual position of the point of view and providing, tothe display system, the portion of the virtual vehicle visible from thevirtual position of the point of view consists of displaying the portionof the virtual vehicle visible within the field of view.

In some embodiments, the method may facilitate a competition between twovirtual vehicles on a physical racecourse. In a scenario where twovirtual vehicles compete on a physical racecourse without any physicalvehicles, then the step of “identifying a position of a physicalvehicle” would be unnecessary. The method could include identifying aposition of a point of view at the racecourse and providing, to adisplay system, a portion of the virtual vehicle visible from theposition of the point of view at the racecourse. All aspects of theforegoing methods not concerning to the position of the physical vehiclecould be applied in such embodiment. In some embodiments, the virtualvehicles are given special properties and a video game appearance. Insome embodiments, video game attributes (i.e., virtual objects) can besimilarly applied to physical vehicles by overlaying those video gameattributes on top of the physical vehicles. For example, cars can begiven boosts, machine guns, missiles (other graphical virtual objectsput into the real world view), virtual jumps, etc. Viewers at theracecourse and at home could view the virtual competitors on thephysical racecourse as if competing in the real-world.

In some embodiments, a method for displaying a virtual vehicle includesmeans for identifying a position of a physical vehicle at a racecourse,means for identifying a position of a point of view at the racecourse,and means for providing, to a display system, a portion of the virtualvehicle visible from a virtual position of the point of view calculatedwithin a virtual world based on the position of the point of view at theracecourse. Problems solved by embodiments disclosed herein can includeovercoming the lack of realism experienced by users of prior solutions.In some embodiments herein, providing visible portions of the virtualvehicle to the user increases the realism experienced by the user. Theincreased realism provides a reliable and re-producible user experienceby providing a real-world race that includes a virtual vehicle.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view includes a portion of the virtualvehicle that is unobscured, from the virtual position of the point ofview, by a representation of the physical vehicle at a virtual positionof the physical vehicle in the virtual world. Problems solved byembodiments disclosed herein can include how to provide a visibleportion of a virtual vehicle. In some embodiments herein, providingvisible portions of the virtual vehicle through a virtual calculation ofthe visible portion increases the accuracy of the visible portiondetermination. The increased accuracy provides a reliable andre-producible user experience by providing a real-world race thatincludes a virtual vehicle. In some embodiments herein, providingvisible portions through a virtual calculation increases the efficiencyof the calculation. The increased efficiency reduces power usage andimproves representation speed for a more seamless user experience.

In some embodiments, the method further includes means for simulating,by the simulation system, an interaction between the virtual vehicle andthe representation of the physical vehicle in the virtual world, theportion of the virtual vehicle visible from the virtual position of thepoint of view is calculated based on the interaction.

In some embodiments, the position of the point of view at the racecourseincludes a point of view of an operator of the physical vehicle, andmeans for identifying a position of a point of view at the racecourseincludes means for detecting, at a sensor, the point of view of theoperator of the physical vehicle, the method further including: meansfor identifying a position of a physical object; means for receivingkinematics information of the virtual vehicle; means for generating, ata display system, a representation of the virtual vehicle based on theposition of the physical object, the position of the point of view atthe racecourse, and the kinematics information; and means for displayingthe representation of the virtual vehicle such that the virtual vehicleis aligned with the physical object from the perspective of the positionof the point of view at the racecourse. Problems solved by embodimentsdisclosed herein can include how to calculate a visible portion of avirtual vehicle. In some embodiments herein, calculating the visibleportion of the virtual vehicle in a virtual world increases the accuracyof the visible portion determination. The increased accuracy provides areliable and re-producible user experience by providing a real-worldrace that includes a virtual vehicle. In some embodiments herein,providing visible portions through a virtual calculation increases theefficiency of the calculation. The increased efficiency reduces powerusage and improves representation speed for a more seamless userexperience.

In some embodiments, the method further includes means for generating,at a display system, the representation of the portion of the virtualvehicle visible from the virtual position of the point of view.

In some embodiments, the method further includes means for displaying,by the display system, a series of representations of the virtualvehicle over a period of time to simulate a trajectory of the virtualvehicle on the racecourse, the series of representations includes therepresentation of the portion of the virtual vehicle visible form thevirtual position of the point of view. In some embodiments, a predictedtrajectory of the virtual vehicle is displayed. The prediction may bebased on current trajectory, acceleration, current vehicle parameters,etc. This may allow an audience member to anticipate if a virtualvehicle is likely to overtake a physical vehicle. The predictedtrajectory may be presented as a line, such as a yellow line. Otherdisplays may also be included, such as “GOING TO PASS!” or “GOING TOCRASH!”

In some embodiments, the method further includes means for storing, bythe display system, a digital 3-D model of the virtual vehicle used togenerate each representation from the series of representations, eachrepresentation is generated by the display system based on the digital3-D model.

In some embodiments, the method further includes means for receiving adigital 3-D model of the virtual vehicle used to generate eachrepresentation from the series of representations, each representationis generated by the display system based on the digital 3-D model.

In some embodiments, the kinematics information includes one or morevectors of motion, one or more scalars of motion, a position vector, aGPS location, a velocity, an acceleration, an orientation, or acombination thereof of the virtual vehicle.

In some embodiments, means for identifying the position of the physicalvehicle includes means for detecting one or more vectors of motion, oneor more scalars of motion, a position vector, a GPS location, avelocity, an acceleration, an orientation, or a combination thereof ofthe virtual vehicle.

In some embodiments, means for identifying the position of the point ofview at the racecourse includes means for detecting a spatial positionof a head of an operator of the physical vehicle. In some embodiments,the method further includes means for transmitting, by a telemetrysystem coupled to the physical vehicle, the spatial position to asimulator system; means for receiving, at the telemetry system,information related to the portion of the virtual vehicle visible fromthe virtual position of the point of view; and means for displaying, tothe operator of the physical vehicle, the representation of the portionof the virtual vehicle based on the information.

In some embodiments, the method further includes means for displayingthe representation of the portion of the virtual vehicle includes: meansfor translating the information into a set of graphical elements, meansfor displaying the representation of the portion includes means fordisplaying the set of graphical elements. In some embodiments, themethod further includes means for computing, at the simulation system,the information related to the portion visible from the virtual positionof the point of view.

In some embodiments, means for displaying the series of representationsof the virtual vehicle includes means for displaying the series ofrepresentation on a display of the physical vehicle, and the display isa transparent organic light-emitting diode (T-OLED) display that allowslight to pass through the T-OLED to display the field of view to theoperator.

In some embodiments, means for displaying the series of representationsof the virtual vehicle includes means for displaying the series ofrepresentations on a display of the physical vehicle, and the display isan LCD display, the method further including: means for capturing, by acamera coupled to the physical vehicle, an image representing the fieldof view of the physical world as seen by the operator on the display inthe physical vehicle; and means for outputting the image on a side ofthe LCD display to display the field of view to the operator, the seriesof representations are overlaid on the image displayed by the LCDdisplay.

In some embodiments, means for displaying the series of representationsof the virtual vehicle includes means for displaying the series ofrepresentations on a display of the physical vehicle, and the displayincludes a front windshield of the physical vehicle, one or more sidewindows of the physical vehicle, a rear windshield of the physicalvehicle, one or more side mirrors, a rearview mirror, or a combinationthereof.

In some embodiments, means for displaying the series of representationsof the virtual vehicle includes means for displaying the series ofrepresentations on a display of a headset worn by the operator. In someembodiments, the headset is a helmet.

In some embodiments, means for identifying the position of the point ofview at the racecourse includes means for detecting one or more of aspatial position of a user's eyes, a gaze direction of the user's eyes,or a focus point of the user's eyes.

In some embodiments, the method further includes: means for providingthe position of the physical vehicle and the position of the point ofview at the racecourse to a simulation system; means for calculating, bythe simulation system, a virtual world including the virtual vehicle anda representation of the physical vehicle; means for calculating, by thesimulation system, a virtual position of the point of view within thevirtual world based on the position of the point of view at theracecourse; and means for calculating, by the simulation system, theportion of the virtual vehicle visible from the virtual position of thepoint of view, and means for providing, to a display system, the portionof the virtual vehicle visible from the virtual position of the point ofview includes means for outputting, by the simulation system, theportion of the virtual vehicle visible from the virtual position of thepoint of view.

In some embodiments, means for identifying the position of the physicalvehicle includes means for receiving a location of each of two portionsof the vehicle.

In some embodiments, means for identifying the position of the physicalvehicle includes means for receiving a location of one portion of thevehicle and an orientation of the vehicle. In some embodiments, meansfor receiving the orientation of the vehicle includes means forreceiving gyroscope data. Problems solved by embodiments disclosedherein can include how to correctly position a physical vehicle in avirtual world for determining a visible portion of a virtual vehicle. Insome embodiments herein, using a measure of orientation provides foraccurate placement of the physical vehicle in the virtual world. Theincreased accuracy provides for a more faithful display of the visibleportions of the vehicle, thereby improving the user experience.

In some embodiments, the position of the point of view at the racecourseincludes a position of a point of view of an operator of the physicalvehicle at the racecourse. In some embodiments, the position of thepoint of view at the racecourse includes a position of a point of viewof an audience member present at a racecourse and observing the physicalvehicle on the racecourse. In some embodiments, the position of thepoint of view at the racecourse includes a position of a camera presentat a racecourse and the method further includes means for imaging thephysical vehicle on the racecourse. In some embodiments, the cameraimages a portion of the racecourse on which the physical vehicle isracing. When the physical vehicle is travelling across the portion ofthe racecourse being captured by the camera, the camera may capture thephysical vehicle in its video feed. When the physical vehicle is nottravelling across the portion of the racecourse being captured by thecamera, the camera may still capture the portion of the racecourse.

In some embodiments, means for identifying the position of the point ofview at the racecourse includes at least one of means for measuring apoint of gaze of eyes, means for tracking eye movement, means fortracking head position, means for identifying a vector from one or botheyes to a fixed point on the physical vehicle, means for identifying avector from a point on the head to a fixed point on the physicalvehicle, means for identifying a vector from a point on eye-wear to afixed point on the physical vehicle, means for identifying a vector froma point on a head gear to a fixed point on the physical vehicle, meansfor identifying a vector from one or both eyes to a fixed point in avenue, means for identifying a vector from a point on the head to afixed point in the venue, means for identifying a vector from a point oneye-wear to a fixed point in the venue, or means for identifying avector from a point on a head gear to a fixed point in the venue. Insome embodiments, means for identifying the position of the point ofview at the racecourse includes means for measuring the point of gaze ofthe eyes and the means for measuring includes means for measuring lightreflection or refraction from the eyes.

In some embodiments, means for providing the position of the physicalvehicle and the position of the point of view at the racecourse includesmeans for wireless transmitting at least one position.

In some embodiments, means for calculating a virtual world includesmeans for transforming physical coordinates of the physical vehicle tocoordinates in the virtual world and the virtual position of thephysical vehicle includes the virtual coordinates.

In some embodiments, means for calculating the portion of the virtualvehicle visible from the virtual position of the point of view includes:means for calculating a representation of the physical vehicle in thevirtual world, means for calculating a representation of a physicalobject in the virtual world between the point of view and the virtualvehicle within the virtual world, and means for extracting a portion ofthe virtual vehicle that is unobscured, from the virtual position of thepoint of view, by the representation of the physical vehicle and therepresentation of the physical object. In some embodiments, the portionof the virtual vehicle within the virtual world that is visible from thevirtual position of the point of view includes the unobscured portion.Problems solved by embodiments disclosed herein can include how tocalculate a visible portion of a virtual vehicle, including more thanjust the portion that is not obscured by the physical vehicle. In someembodiments herein, calculating the visible portion in a virtual worldthat includes physical objects in the real world increases the accuracyof the visible portion determination. The increased accuracy provides areliable and re-producible user experience by providing a real-worldrace that includes a virtual vehicle. In some embodiments herein,providing visible portions through a virtual calculation increases theefficiency of the calculation. The increased efficiency reduces powerusage and improves representation speed for a more seamless userexperience.

In some embodiments, means for extracting the portions of the virtualvehicle may include means for determining which pixels are obstructed byother representations, and only displaying pixels that are notobstructed by other representations. In some embodiments, means forextracting the portions of the virtual vehicle may include means forsetting a pixel alpha value of zero percent (in RGBA space) for allpixels obstructed by other representations. For example, portions of thevirtual vehicle may be obstructed by other virtual representations,e.g., another virtual vehicle, or representations of physical objects,e.g., objects within a physical vehicle or the physical vehicle itself.Any observed (from the virtual position of the point of view) pixelvalues can be used to provide the portions of the virtual vehicle thatare visible from the virtual position of the point of view. In someembodiments, the pixels of unobscured and observed portions of thevirtual vehicle can each be set to include an alpha value greater thanzero percent (in RGBA space) to indicate that those unobscured pixelscan be seen and should be displayed. In contrast, pixels set to an alphavalue of zero percent indicate that those pixels are fully transparent,i.e., invisible, and would not be displayed.

In some embodiments, means for calculating the representation of thephysical object between the virtual position of the point of view andthe representation of the physical vehicle includes means for accessinga database of representations to obtain a virtual position of thephysical object.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view consists of portions of thevirtual vehicle that are unobscured by other representations in thevirtual world.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view includes a virtual shadow in thevirtual world. In some embodiments, the virtual shadow is at least oneof a shadow projected by the virtual vehicle and a shadow projected ontothe virtual vehicle. In some embodiments, the portion of the virtualvehicle visible from the position of the point of view at the racecourseincludes a virtual reflection. In some embodiments, the virtualreflection is at least one of a reflection of the virtual vehicle and areflection on the virtual vehicle.

In some embodiments, means for calculating, by the simulation system, aportion of the virtual vehicle within the virtual world that is visiblefrom the virtual position of the point of view includes means forcalculating a field of view from the virtual position of the point ofview and means for providing, to the display system, the portion of thevirtual vehicle visible from the virtual position of the point of viewincludes displaying the portion of the virtual vehicle within the fieldof view.

In some embodiments, means for calculating, by the simulation system, aportion of the virtual vehicle within the virtual world that is visiblefrom the position of the virtual point of view includes means forcalculating a field of view from the virtual position of the point ofview and means for providing, to the display system, the portion of thevirtual vehicle visible from the virtual position of the point of viewconsists of means for displaying the portion of the virtual vehiclewithin the field of view.

In some embodiments, a system for displaying virtual vehicles includes afirst sensor detecting a position of a physical vehicle at a racecourse,a second sensor detecting a position of a point of view at theracecourse, and a simulation system outputting a portion of the virtualvehicle visible from a virtual position of the point of view. Problemssolved by embodiments disclosed herein can include overcoming the lackof realism experienced by users of prior solutions. In some embodimentsherein, providing visible portions of the virtual vehicle to the userincreases the realism experienced by the user. The increased realismprovides a reliable and re-producible user experience by providing areal-world race that includes a virtual vehicle.

In some embodiments, the simulation system determines the visibleportion of the virtual vehicle based on a virtual position of thephysical vehicle in a virtual world, a virtual position of the point ofview in the virtual world, and a virtual position of the virtual vehiclein the virtual world. Problems solved by embodiments disclosed hereincan include how to provide a visible portion of a virtual vehicle. Insome embodiments herein, providing visible portions of the virtualvehicle through a virtual calculation of the visible portion increasesthe accuracy of the visible portion determination. The increasedaccuracy provides a reliable and re-producible user experience byproviding a real-world race that includes a virtual vehicle. In someembodiments herein, providing visible portions through a virtualcalculation increases the efficiency of the calculation. The increasedefficiency reduces power usage and improves representation speed for amore seamless user experience.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view includes a portion of the virtualvehicle that is unobscured, from the virtual position of the point ofview, by a representation of the physical vehicle at the virtualposition of the physical vehicle.

In some embodiments, the system further includes the simulation systemconfigured to simulate an interaction between the virtual vehicle andthe representation of the physical vehicle in the virtual world, theportion of the virtual vehicle visible from the virtual position of thepoint of view is calculated based on the interaction.

In some embodiments, the system includes a simulation system configuredto: receive the position of the physical vehicle and the position of thepoint of view at the racecourse; calculate a virtual world including thevirtual vehicle and a representation of the physical vehicle; calculatea virtual position of the point of view within the virtual world basedon the position of the point of view at the racecourse; calculate theportion of the virtual vehicle visible from the virtual position of thepoint of view; and output, to the display system, the portion of thevirtual vehicle visible from the virtual position of the point of view.Problems solved by embodiments disclosed herein can include how tocalculate a visible portion of a virtual vehicle. In some embodimentsherein, calculating the visible portion of the virtual vehicle in avirtual world increases the accuracy of the visible portiondetermination. The increased accuracy provides a reliable andre-producible user experience by providing a real-world race thatincludes a virtual vehicle. In some embodiments herein, providingvisible portions through a virtual calculation increases the efficiencyof the calculation. The increased efficiency reduces power usage andimproves representation speed for a more seamless user experience.

In some embodiments, the first sensor receives a location of each of twoportions of the vehicle. In some embodiments, the first sensor receivesa location of one portion of the vehicle and an orientation of thevehicle. In some embodiments, receiving the orientation of the vehicleincludes receiving gyroscope data. Problems solved by embodimentsdisclosed herein can include how to correctly position a physicalvehicle in a virtual world for determining a visible portion of avirtual vehicle. In some embodiments herein, using a measure oforientation provides for accurate placement of the physical vehicle inthe virtual world. The increased accuracy provides for a more faithfuldisplay of the visible portions of the vehicle, thereby improving theuser experience.

In some embodiments, the position of the point of view at the racecourseincludes a position of a point of view of an operator of the physicalvehicle at the racecourse. In some embodiments, the position of thepoint of view at the racecourse includes a position of a point of viewof an audience member present at a racecourse and observing the physicalvehicle on the racecourse.

In some embodiments, the position of the point of view at the racecourseincludes a position of a camera present at a racecourse and imaging thephysical vehicle on the racecourse. In some embodiments, the cameraimages a portion of the racecourse on which the physical vehicle isracing. When the physical vehicle is travelling across the portion ofthe racecourse being captured by the camera, the camera may capture thephysical vehicle in its video feed. When the physical vehicle is nottravelling across the portion of the racecourse being captured by thecamera, the camera may still capture the portion of the racecourse.

In some embodiments, the second sensor is configured to detect theposition of the point of view at the racecourse by at least one ofmeasuring the point of gaze of eyes, tracking eye movement, trackinghead position, identifying a vector from one or both eyes to a fixedpoint on the physical vehicle, identifying a vector from a point on thehead to a fixed point on the physical vehicle, identifying a vector froma point on eye-wear to a fixed point on the physical vehicle,identifying a vector from a point on a head gear to a fixed point on thephysical vehicle, identifying a vector from one or both eyes to a fixedpoint in a venue, identifying a vector from a point on the head to afixed point in the venue, identifying a vector from a point on eye-wearto a fixed point in the venue, or identifying a vector from a point on ahead gear to a fixed point in the venue. In some embodiments, secondsensor is configured to detect the position of the point of view at theracecourse by measuring light reflection or refraction from the eyes.

In some embodiments, receiving the position of the physical vehicle andthe position of the point of view at the racecourse includes wirelessreceiving at least one position.

In some embodiments, calculating a virtual world includes transformingphysical coordinates of the physical vehicle to coordinates in thevirtual world and the virtual position of the physical vehicle includesthe virtual coordinates.

In some embodiments, calculating a portion of the virtual vehiclevisible from the position of the point of view at the racecourseincludes: calculating a representation of the physical vehicle in thevirtual world, calculating a representation of a physical object in thevirtual world between the point of view and the virtual vehicle withinthe virtual world, and extracting a portion of the virtual vehicle thatis unobscured, from the virtual position of the point of view, by therepresentation of the physical vehicle and the representation of thephysical object. In some embodiments, the portion of the virtual vehiclewithin the virtual world that is visible from the virtual position ofthe point of view includes the unobscured portion. Problems solved byembodiments disclosed herein can include how to calculate a visibleportion of a virtual vehicle, including more than just the portion thatis not obscured by the physical vehicle. In some embodiments herein,calculating the visible portion in a virtual world that includesphysical objects in the real world increases the accuracy of the visibleportion determination. The increased accuracy provides a reliable andre-producible user experience by providing a real-world race thatincludes a virtual vehicle. In some embodiments herein, providingvisible portions through a virtual calculation increases the efficiencyof the calculation. The increased efficiency reduces power usage andimproves representation speed for a more seamless user experience.

In some embodiments, extracting the portions of the virtual vehicle mayinclude determining which pixels are obstructed by otherrepresentations, and only displaying pixels that are not obstructed byother representations. In some embodiments, extracting the portions ofthe virtual vehicle may include setting a pixel alpha value of zeropercent (in RGBA space) for all pixels obstructed by otherrepresentations. For example, portions of the virtual vehicle may beobstructed by other virtual representations, e.g., another virtualvehicle, or representations of physical objects, e.g., objects within aphysical vehicle or the physical vehicle itself. Any observed (from thevirtual position of the point of view) pixel values can be used toprovide the portions of the virtual vehicle that are visible from thevirtual position of the point of view. In some embodiments, the pixelsof unobscured and observed portions of the virtual vehicle can each beset to include an alpha value greater than zero percent (in RGBA space)to indicate that those unobscured pixels can be seen and should bedisplayed. In contrast, pixels set to an alpha value of zero percentindicate that those pixels are fully transparent, i.e., invisible, andwould not be displayed.

In some embodiments, calculating the representation of the physicalobject between the virtual position of the point of view and therepresentation of the physical vehicle includes accessing a database ofrepresentations to obtain a virtual position of the physical object.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view consists of portions of thevirtual vehicle that are unobscured by other representations in thevirtual world.

In some embodiments, the portion of the virtual vehicle visible from thevirtual position of the point of view includes a virtual shadow in thevirtual world. In some embodiments, the virtual shadow is at least oneof a shadow projected by the virtual vehicle and a shadow projected ontothe virtual vehicle. In some embodiments, the portion of the virtualvehicle visible from the virtual position of the point of view includesa virtual reflection. In some embodiments, the virtual reflection is atleast one of a reflection of the virtual vehicle and a reflection on thevirtual vehicle.

In some embodiments, calculating a portion of the virtual vehicle withinthe virtual world that is visible from the virtual position of the pointof view includes calculating a field of view from the virtual positionof the point of view and providing, to the display system, the portionof the virtual vehicle visible from the virtual position of the point ofview includes displaying the portion of the virtual vehicle within thefield of view.

In some embodiments, calculating a portion of the virtual vehicle withinthe virtual world that is visible from the position of the point of viewat the racecourse includes calculating a field of view from the virtualposition of the point of view and providing, to the display system, theportion of the virtual vehicle visible from the virtual position of thepoint of view consists of displaying the portion of the virtual vehiclevisible within the field of view.

In some embodiments, the system may facilitate a competition between twovirtual vehicles on a physical racecourse. In a scenario where twovirtual vehicles compete on a physical racecourse without any physicalvehicles, then the first sensor detecting a position of a physicalvehicle would unnecessary. The system in such an embodiment couldinclude a sensor detecting a position of a point of view at theracecourse and a display system providing a portion of the virtualvehicle visible from the position of the point of view at theracecourse. All aspects of the foregoing systems not concerning to theposition of the physical vehicle could be applied in such embodiment. Insome embodiments, the virtual vehicles are given special properties anda video game appearance. For example, cars can be given boosts, machineguns, missiles (other graphical virtual objects put into the real worldview), virtual jumps, etc. In some embodiments, the physical vehiclescan be given similar video game attributes. For example, graphic virtualobjects such as machine guns or missiles etc. may be rendered andoverlaid on the physical vehicles as observed on a display. Viewers atthe racecourse and at home could view the virtual competitors on thephysical racecourse as if competing in the real-world.

As used herein, “point of view” can be understood to be a real worldposition from which the virtual vehicle will be viewed. For example, anoperator of a physical vehicle (e.g., a driver in the physical vehicle)viewing his surroundings. A display may be used to augmenting theoperator's view by introducing the virtual vehicle in the view. Becausethe virtual vehicle is added to the real world point of view, if thedisplay system was not provided or discontinued, the real world point ofview would not see a virtual vehicle.

FIG. 1 is a diagram 100 of a physical vehicle 101, according to someembodiments. FIG. 1 provides an example of a point of view of anoperator (e.g., driver) of physical vehicle 101. Thus, the position ofthe point of the view of the operator is the position and direction(gaze) of the operator's eyes (or some approximation of the operator'seye position and direction). Although FIG. 1 is provided with referenceto the point of view of the operator of a physical vehicle, theteachings apply equally to other points of view, such as audience memberat a racecourse on which physical vehicle is driving or a camera at theracecourse.

Physical vehicle 101 includes a display system 102 (including renderingcomponent 107), a simulation component 106, a telemetry system 104(including sensors 108), RF circuitry 105, and a force controller 112.Physical vehicle 101 also includes eye-position detector 110, frontwindshield 120, rear-view mirror 122, rear windshield 124, side windows126A and 126B, side mirrors 128A and 128B, seat and head brace 130,speakers 132, and brakes 134. FIG. 1 also includes a vehicle operator114. In FIG. 1, vehicle operator 114 is illustrated wearing a helmet116, a visor over eyes 117, and a haptic suit 118. In some embodiments,the visor worn over eyes 117 is a component of helmet 116.

As shown in FIG. 1, physical vehicle 101 is an automobile. In someembodiments, devices within physical vehicle 101 communicate with asimulation system 140 to simulate one or more virtual vehicles within afield of view of vehicle operator 114. Simulation system 140 may beon-board physical vehicle 101. In some embodiments, as illustrated indiagram 100, simulation system 140 may be remote from physical vehicle101, also described elsewhere herein. In some embodiments, thefunctionality performed by simulation system 140 may be distributedacross both systems that are on-board physical vehicle 101 and remotefrom physical vehicle 101. In some embodiments, simulation system 140generates and maintains a racing simulation 141 between one or more liveparticipants (i.e., vehicle operator 114 operating physical vehicle 101)and one or more remote participants (not shown).

Simulating virtual vehicles in real-time enhances the racing experienceof vehicle operator 114. Implementing simulation capabilities withinphysical vehicle 101 allows vehicle operator 114, who is a liveparticipant, to compete against a remote participant operating a virtualvehicle within racing simulation 141. The field of view of vehicleoperator 114 is the observable world seen by vehicle operator 114augmented with a virtual vehicle. In some embodiments, the augmentationcan be provided by a display, e.g., displays housed in or combined withone or more of front windshield 120, rear-view mirror 122, rearwindshield 124, side windows 126A and 126B, and side mirrors 128A and128B.

In some embodiments, the augmentation can be provided by a hologramdevice or 3-D display system. In these embodiments, front windshield120, rear-view mirror 122, rear windshield 124, side windows 126A and126B, or side mirrors 128A and 128B can be T-OLED displays that enable3-D images to be displayed, utilizing cameras to capture thesurroundings displayed with 3-D images overlaid on non-transparentdisplays.

In some embodiments, the augmentation can be provided by a head-mounteddisplay (HMD) worn by vehicle operator 114 over eyes 117. The HMD may beworn as part of helmet 116. In some embodiments, the HMD is imbedded invisors, glasses, goggles, or other devices worn in front of the eyes ofvehicle operator 114. Like the display described above, the HMD mayoperate to augment the field of view of vehicle operator 114 byrendering one or more virtual vehicles on one or more displays in theHMD.

In other embodiments, the HMD implements retinal projection techniquesto simulate one or more virtual vehicles. For examples, the HMD mayinclude a virtual retinal display (VRD) that projects images onto theleft and right eyes of vehicle operator 114 to create athree-dimensional (3D) image of one or more virtual vehicles in thefield of view of vehicle operator 114.

In some embodiments, the augmentation can be provided by one or moredisplays housed in physical vehicle 101 as described above (e.g., frontwindshield 120 and rear-view mirror 122), a display worn by vehicleoperator 114 as described above (e.g., an HMD), a hologram device asdescribed above, or a combination thereof. An advantage of simulatingvirtual vehicles on multiple types of displays (e.g., on a displayhoused in physical vehicle 101 and an HMD worn by vehicle operator 114)is that an augmented-reality experience can be maintained when vehicleoperator 114 takes off his HMD. Additionally, multiple participants inphysical vehicle 101 can share the augmented-reality experienceregardless of whether each participant is wearing an HMD.

In some embodiments, multiple virtual vehicles are simulated for vehicleoperator 114. For example, multiple virtual vehicles are displayed infront and/or behind and/or beside the operator. For example, one or morevirtual vehicles may be displayed on front windshield 120 (an exampledisplay) and one or more virtual vehicles may be displayed on rear-viewmirror 122 (an example display). In addition, one or more virtualvehicles may be displayed on the HMD. Similarly, physical vehicle 101may be one of a plurality of physical vehicles in proximity to eachother. In some embodiments, a virtual vehicle being simulated forvehicle operator 114 can be another physical vehicle running on aphysical racecourse at a different physical location than that being runby vehicle operator 114. For example, one driver could operate a vehicleon a racecourse in Monaco and another driver could operate a vehicle onreplica-racecourse in Los Angeles. Embodiments herein contemplatepresenting one or both of the Monaco and Los Angeles driver with avirtual vehicle representing the other driver.

Returning to simulation system 140 and as described above, simulationsystem 140 may include racing simulation 141, which simulates acompetition between physical vehicle 101 and one or more virtualvehicles on a virtual racecourse. In some embodiments, the virtualracecourse is generated and stored by simulation system 140 tocorrespond to the physical racecourse in which vehicle operator 114 isoperating, e.g., driving, physical vehicle 101. In some embodiments, thevirtual racecourse is generated using 360 degree laser scan videorecording or similar technology. Therefore, as vehicle operator 114controls physical vehicle 101 on the physical racecourse in real-time,the virtual trajectory of physical vehicle 101 within racing simulation141 is simulated by simulation system 140 to emulate the physical,real-world trajectory of physical vehicle 101 on the physicalracecourse.

In some embodiments, to enable simulation system 140 to simulatephysical vehicle 101 on the virtual racecourse in racing simulation 141,physical vehicle 101 includes telemetry system 104. Telemetry system 104includes sensors 108 that detect data associated with physical vehicle101. Sensors 108 include one or more devices that detect kinematicsinformation of physical vehicle 101. In some embodiments, kinematicsinformation includes one or more vectors of motion, one or more scalarsof motion, an orientation, a Global Positioning System (GPS) location,or a combination thereof. For example, a vector of motion may include avelocity, a position vector, or an acceleration. For example, a scalarof motion may include a speed. Accordingly, sensors 108 may include oneor more accelerometers to detect acceleration, one or more GPS (orGLONASS or other global navigation system) receiver to detect the GPSlocation, one or more motion sensors, one or more orientation sensors,or a combination thereof. In some embodiments, the real-time datacollected by sensors 108 are transmitted to simulation system 140. Otherreal-time data may include measurements of the car, heat, tiretemperature, etc. In some embodiments, one or more of the kinematicsinformation and car measurements are used for simulation predictability.For example, some embodiments may include predictive simulation enginesthat pre build scenes based on these other measurements and the velocityand acceleration information.

In some embodiments, physical vehicle 101 includes radio frequency (RF)circuitry 105 for transmitting data, e.g., telemetric informationgenerated by telemetry system 104, to simulation system 140. RFcircuitry 105 receives and sends RF signals, also called electromagneticsignals. RF circuitry 105 converts electrical signals to/fromelectromagnetic signals and communicates with communications networksand other communications devices via the electromagnetic signals. RFcircuitry 105 may include well-known circuitry for performing thesefunctions, including but not limited to an antenna system, an RFtransceiver, one or more amplifiers, a tuner, one or more oscillators, adigital signal processor, a CODEC chipset, a subscriber identity module(SIM) card, memory, and so forth. RF circuitry 105 may communicate withnetworks, such as the Internet, also referred to as the World Wide Web(WWW), an intranet and/or a wireless network, such as a cellulartelephone network, a wireless local area network (LAN) and/or ametropolitan area network (MAN), and other devices by wirelesscommunication. The wireless communication optionally uses any of aplurality of communications standards, protocols and technologies,including but not limited to Global System for Mobile Communications(GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packetaccess (HSDPA), high-speed uplink packet access (HSDPA), Evolution,Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long termevolution (LTE), near field communication (NFC), wideband code divisionmultiple access (W-CDMA), code division multiple access (CDMA), timedivision multiple access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE),Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE802.11g and/or IEEE 802.11n), Wi-MAX, a protocol for e-mail (e.g.,Internet message access protocol (IMAP) and/or post office protocol(POP)), instant messaging (e.g., extensible messaging and presenceprotocol (XMPP), Session Initiation Protocol for Instant Messaging andPresence Leveraging Extensions (SIMPLE), Instant Messaging and PresenceService (IMPS)), and/or Short Message Service (SMS), or any othersuitable communication protocol.

In some embodiments, simulation system 140 includes RF circuitry,similar to RF circuitry 105, for receiving data from physical vehicle101. Based on the telemetric information received from physical vehicle101, simulation system 140 simulates physical vehicle 101 as an avatarwithin racing simulation 141. In some embodiments, as will be furtherdescribed with respect to FIG. 5, simulation system 140 receives inputsfor controlling and simulating one or more virtual vehicles withinracing simulation. In some embodiments, simulation system 140 calculateskinematics information of the virtual vehicle based on the receivedinputs and a current state of the virtual vehicle on the virtualracecourse in racing simulation 141. For example, the current state mayrefer to a coordinate, a position, a speed, a velocity, an acceleration,an orientation, etc., of the virtual vehicle being simulated on thevirtual racecourse. To replicate the virtual race between a liveparticipant, i.e., vehicle operator 114, and a virtual participantoperating a virtual vehicle for vehicle operator 114, simulation system140 transmits kinematics information of the virtual vehicle tocomponents (e.g., display system 102 or simulation component 106) inphysical vehicle 101 via RF circuitry. As described elsewhere in thisdisclosure, it is to be understood that depending on the type of contextbeing simulated, the display system 102 and other components shown inphysical vehicle 101 may be housed in other types of devices.

In some embodiments, to enable two-way interactive racing whereinteractions simulated in racing simulation 141 can be reproduced forvehicle operator 114 driving physical vehicle 101, simulation system 140determines whether the avatar of physical vehicle 101 within racingsimulation 141 is in contact with obstacles, such as the virtualvehicle, being simulated within racing simulation 141. Upon determininga contact, simulation system 140 calculates force information, audioinformation, or a combination thereof associated with the contact. Insome embodiments, simulation system 140 transmits the force or audioinformation to physical vehicle 101 where the force or audio informationis reproduced at physical vehicle 101 to enhance the virtual realityracing experience for vehicle operator 114.

Returning to physical vehicle 101, physical vehicle 101 includes adisplay system 102 for generating a representation of a virtual vehiclebased on information, e.g., kinematics information of the virtualvehicle, received from simulation system 140 via RF circuitry 105. Insome embodiments, display system 102 is coupled to a simulationcomponent 106 that generates a virtual representation of the virtualvehicle based on the kinematics information of the virtual vehiclereceived from simulation system 140. In some embodiments, simulationcomponent 106 generates the virtual representation based on thekinematics information and eyes measurements (e.g., a spatial position)of the eyes 117 of vehicle operator 114. To further enhance the realismof the virtual representation, i.e., a graphically generated vehicle,simulation component 106 generates the virtual representation based onthe kinematics information, a spatial position of eyes 117 of vehicleoperator 114, a gaze direction of eyes 117, and a focus point of eyes117, according to some embodiments. As described herein, eyesmeasurements may include a spatial position of eyes 117, a gazedirection of eyes 117, a focus point of eyes 117, or a combinationthereof of the left eye, the right eye, or both left and right eyes.

In some embodiments, physical vehicle 101 includes eye-position detector110, e.g., a camera or light (e.g., infrared) reflection detector, todetect eyes measurements (e.g., spatial position, gaze direction, focuspoint, or a combination thereof of eyes 117) of eyes 117 of vehicleoperator 114. In some embodiments, eye-position detector 110 detects aspatial position of the head of vehicle operator 114 to estimate themeasurements of eyes 117. For example, eye-position detector 110 maydetect helmet 116 or a visor on helmet 116 to estimate the measurementsof eyes 117. Detecting eye measurements and/or head position may alsoinclude detecting at least one of a position and an orientation ofhelmet 116 worn by vehicle operator 114.

The position of the eyes may be calculated directly (e.g., from a fixedsensor, such as a track-side sensor) or based on a combination of thesensor in the car and a position of the car.

In some embodiments, eye-position detector 110 includes a camera thatcan record a real-time video sequence or capture a series of images ofthe face of vehicle operator 114. Then, eye-position detector 110 maytrack and detect the measurements of eyes 117 by analyzing the real-timevideo sequence or the series of images. In some embodiments,eye-position detector 110 implements one or more algorithms for trackinga head movement or orientation of vehicle operator 114 to aid in eyetracking, detection, and measurement. As shown in diagram 100,eye-position detector 110 may be coupled to rear-view mirror 122.However, as long as eye-position detector 110 is implemented inproximity to physical vehicle, eye-position detector 110 can be placedin other locations, e.g., on the dashboard, inside or outside physicalvehicle 101. In some embodiments, to increase accuracy in detection,eye-position detector 110 can be implemented within helmet 116 or otherhead-mounted displays (HMDs), e.g., visors or goggles, worn by vehicleoperator 114 over eyes 117.

In some embodiments, eye-position detector 110 implemented within a HMDfurther includes one or more focus-tunable lenses, one or moremechanically actuated displays, and mobile gaze-tracking technologyreproduced scenes can be drawn that actually correct common refractiveerrors in the VR world as the eyes are continually monitored based onwhere a user looks in a virtual scene. An advantage of the abovetechniques is that vehicle operator 114 would not need to wear contactlenses or corrective glasses while wearing a HMD implanting eye-positiondetector 110.

In some embodiments, simulation component 106 generates the virtualrepresentation of the virtual vehicle based on a 3-D model of thevirtual vehicle. For example, the virtual representation may represent aperspective of the 3-D model as viewed from eyes 117. In someembodiments, by generating the virtual representation from theperspective of eyes 117 whose measurements are detected or estimated byeye-position detector 110, the virtual representation can be simulatedin accurate dimensions and scaling for vehicle operator 114 to increasethe reality of racing against the virtual vehicle. In some embodiments,the 3-D model may be pre-stored on simulation component 106 or receivedfrom simulation system 140.

In some embodiments, rendering component 107 within display system 102displays the generated virtual representation on one or more displays ofphysical vehicle 101. As discussed above, the virtual representation maybe generated by simulation component 106 in some embodiments orsimulation system 140 in some embodiments. The one or more displays mayinclude windows of physical vehicle 101, e.g., front windshield 120 orside windows 126A-B, or mirrors of physical vehicle 101, e.g., rear-viewmirror 122 or side mirrors 128A-B. In some embodiments, the one or moredisplays may be components in helmet 116. Helmet 116 may include ahelmet, visors, glasses, or a goggle system worn by vehicle operator114.

In some embodiments, one or more displays (e.g., front windshield 120)can be transparent organic light-emitting diode (T-OLED) displays thatallow light to pass through the T-OLED to display the field of view tovehicle operator 114. In these embodiments, rendering component 107renders the virtual representation of the virtual vehicle as a layer ofpixels on the one or more displays. The T-OLED displays may allowvehicle operator 114 to see both the simulated virtual vehicle and thephysical, un-simulated world in his field of view.

In other embodiments, one or more displays (e.g., front windshield 120)can be non-transparent liquid crystal displays (LCDs). In theseembodiments, unlike the T-OLED displays, the LCD cannot allow light topass through to enable vehicle operator 114 to see the physical,un-simulated world in his field of view. Instead, in these embodiments,a camera (e.g., a pinhole camera) facing outwards with respect to theLCD and coupled to physical vehicle 101 can record a live video feed ofthe physical, un-simulated world and representing the field of view ofthe physical world as would be seen from eyes 117 of vehicle operator114 if the LCD were to be transparent (e.g., an T-OLED display). Then,rendering component 107 may display the live video feed on the interiorside of the LCD to display the field of view to vehicle operator 114.Further, rendering component 107 may overlay the generated virtualrepresentation on the live video feed being displayed by the LCD toenable vehicle operator 114 to see the simulated, virtual vehicle.

In some embodiments, the non-transparent LCD cannot by itself displayimages or live video feeds in 3D. Accordingly, the camera used to recordthe live video feed may include one or more cameras that are part of astereoscopic camera system to record the physical world in color and in3D. In some embodiments, to further enhance the 3D effect of the livevideo feed being displayed, the non-transparent LCD is a multi-viewautostereoscopic 3D display, i.e., an automultiscopic display, thatenables vehicle operator 114 to view the displayed 3D video feed fromdifferent angles as vehicle operator 114 moves his head and as a result,his eyes 117.

In high speed racing events, the head of vehicle operator 114 moves verylittle. Therefore, in some embodiments, eye-position detector 110 fortracking a position of the head or eyes 117 of vehicle operator 114 maybe omitted from physical vehicle 101 to reduce a number of componentsand complexity in simulating virtual vehicles at physical vehicle 101.In some embodiments, to enable vehicle operator 114 to freely move hishead, eye-position detector 110 is implemented to track a position ofthe head or eyes 117 of vehicle operator 114. In embodiments where thedisplay is a non-transparent display, an angle of one or more cameras,e.g., cameras in a stereoscopic camera system, can be adjusted tocorrespond to the tracked position of the head or eyes 117 of vehicleoperator 114.

In some embodiments, to enable two-way interactive racing whereinteractions simulated on the virtual racecourse in racing simulation141 can be replicated for vehicle operator 114 driving physical vehicle101, simulation component 106 determines a proximity of the virtualvehicle with physical vehicle 101 based on the kinematics information ofthe virtual vehicle received from simulation system 140. In someembodiments, upon determining a contact between the virtual vehicle andphysical vehicle 101 based on the determined proximity, simulationsystem 140 calculates force information, audio information, or acombination thereof associated with the contact. Then, simulationcomponent 106 may transmit the force information to force controller 112and/or the audio information to speakers 132. In some embodiments,playing the audio information via speakers 132 may emulate the sound ofthe engine, brakes, tires, and collision between physical vehicle 101and the virtual vehicle being simulated by simulation system 140. Insome embodiments, the audio information may include a volume that iscalculated based on a distance calculated between physical vehicle 101and the virtual vehicle on the simulated racecourse, and may take intoaccount the directional position of the head and orientation of the earsof vehicle operator 114. In some embodiments, speakers 132 may includeaudio devices equipped in physical vehicle 101 (e.g., a loudspeaker orspeaker system) or audio devices worn by vehicle operator 114 (e.g.,headphones or earpieces). In embodiments where speakers 132 are worn byvehicle operator 114, speakers 132 may be implemented within ahead-mounted display such as helmet 116.

For sound reproduced for observers not in the physical vehicle (e.g., anaudience member at a racecourse or watching at home), speakers can beplaced around the track or through the headgear or audience member orpoint of view of camera. In the virtual world, microphone position canbe set, just like with camera position. Similarly, sound generationspositions can be set. In some embodiments, when the virtual car with asound generation position is further from the microphone the noise itmakes would be less.

In some embodiments, force controller 112 controls one or more actuatorsbased on the force information to emulate the contact simulated betweenphysical vehicle 101 and the virtual vehicle on the virtual racecoursesimulated by simulation system 140. For example, force controller 112may control one or more force actuators built into seat and head brace130 to emulate how vehicle operator 114 would feel, e.g., a sensation ofbeing bumped in the head, in a collision. Similarly, in someembodiments, force controller 112 may communicate via wired or wirelesscommunications with a haptic suit 118 worn by vehicle operator 114 toemulate the sensation of vehicle operator 114 contacting a real physicalvehicle. Force controller 112 may also control one or more forceactuators built into the physical vehicle 101 to emulate the physicalvehicle 101 making contact with a virtual object. In some embodiments,force controller 112 controls one or more mechanical systems that affectthe actual functioning of physical vehicle 101. For example, forcecontroller 112 may control one or more brakes 134, a steering column, ora power of physical vehicle 101 among other mechanical and/or electricalsystems to emulate the effects of contact between physical vehicle 101and the virtual vehicle or virtual object were the virtual object, suchas a virtual vehicle, actually a physical object on the same racecourseas physical vehicle 101. As described above, in some embodiments, thevirtual object being simulated can be a physical vehicle in the realworld on a racecourse at a different physical location. Therefore, thepresent disclosure also enables two vehicle operators running ondifferent racecourses to feel as if they are competing on the sameracecourse since their counterpart physical vehicles can be simulated asvirtual vehicles.

In some embodiments involving electric cars, emulating effects mayinclude control over the power generated to the axels, and in some casesto each specific wheel in a car with four electric motors (one for eachwheel). This may advantageously allow a bump to be simulated by a smallspike down in the motor of the wheel. This can be controlled based onthe duration of the impact and other factors.

In some embodiments, as described with respect to FIG. 5, some or all ofthe functionality of simulation component 106 described above may beperformed remotely by, for example, simulation system 140. In theseembodiments, display system 102 receives a virtual representationgenerated by simulation system 140. Relatedly, in these embodiments,force controller 112 and speakers 132 may receive force and audioinformation, respectively, that is calculated by simulation system 140.

FIGS. 2A-C are diagrams showing how multiple virtual vehicles aredisplayed on one or more displays, respectively, according to someembodiments. For ease of explanation, FIGS. 2A-C will be described withrespect to the elements, e.g., display system 102 and vehicle operator114, of FIG. 1. According to some embodiments, the virtual vehicles canbe displayed according to an augmented reality embodiment or afull-rendering embodiment, each of which will be further describedbelow.

In the augmented reality embodiment, view 202A and 204A of FIG. 2A canbe augmented with virtual vehicles 228 and 230 being output onrespective displays 220 and 222 of FIG. 2C to enable vehicle operator114 to see portions of virtual vehicles 212 and 214 shown in respectiveviews 202B and 204B of FIG. 2B. In some embodiments, display 220 anddisplay 222 may correspond to front windshield 120 and rear-view mirror122 as described with respect to FIG. 1

As shown in FIG. 2A, view 202A shows the field of view of vehicleoperator 114 who may see, via displays 220 and 222, a physical vehicle206A on the physical racecourse. Similarly, view 204A shows empty spaceas there are no real vehicles on the physical racecourse viewable byvehicle operator 114 via display 222.

In some embodiments, to simulate one or more virtual vehicles fordisplay on display 220 and 222, display system 102 identifies one ormore positions 224 and 226 on displays 220 and 222, respectively. Insome embodiments, position 224 may correspond to a physical position208A in the field of view of vehicle operator 114. For example, position224 on display 220 may correspond to a portion of the physicalracecourse or a portion of a building or landmark viewable by vehicleoperator 114 (shown as position 208A in view 202A) and simulated in thevirtual racecourse within racing simulation 141 of simulation system140. Similarly, position 26 may correspond to a different portion of thephysical racecourse or a portion of a building or landmark viewable byvehicle operator 114 (shown as position 210A in view 202A).

In some embodiments, as described with respect to FIG. 1, simulationcomponent 106 may generate a first virtual vehicle 228 as a firstrepresentation and a second virtual vehicle 230 as a secondrepresentation. In some embodiments, simulation component 106 maygenerate the first virtual vehicle 228 based on position 224,measurements of eyes 117 (as described with respect to eye-positiondetector 110), and the kinematics of the first virtual, competitorvehicle. As shown in display 220, rendering component 107 displays firstvirtual vehicle 228 to align with position 224. Similarly, renderingcomponent 107 may display second virtual vehicle 230 to align withposition 226 on display 222.

As a result, vehicle operator 114 may see physical objects and virtualobjects as shown in views 202B and 204B. For example, similar to view202A, view 202B shows that vehicle operator 114 may continue to seephysical vehicle 206B. However, view 202B shows that vehicle operator114 may see virtual objects such as virtual vehicle 212 displayed asvirtual vehicle 228 on display 220. Similarly, view 204B shows thatvehicle operator 114 may see virtual objects such as virtual vehicle 214displayed as virtual vehicle 230 on display 222. Further embodiments aredescribed with respect to FIG. 10.

In the full rendering embodiment, displays 220 and 222 can be configuredto render both physical and virtual objects for display to enablevehicle operator 114 to see virtual vehicles alongside physicalvehicles. In this embodiment, rendering component 107 can render anddisplay physical objects such as roads and physical vehicle 206A asshown in views 202A and 204A of FIG. 2A. As described above with respectto the augmented reality embodiment, simulation component 106 maygenerate a first virtual vehicle 228 as a first representation and asecond virtual vehicle 230 as a second representation for display. Inthe full rendering embodiment, displays 220 and 222 can be configured todisplay virtual vehicles 228 and 230, respectively, alongside physicalobjects as shown in views 202B and 204B seen by vehicle operator 114.For example, view 202B shows the road, physical vehicle 206B, andvirtual vehicle 212 being rendered and displayed. Similarly, view 204Bshows virtual vehicle 214 being rendered and displayed. In someembodiments, in the full rendering embodiment, an outward facing camera(with respect to displays 220 and/or 222) can capture a live video feedof the surroundings. In these embodiments, displays 220 and 222 can beconfigured to display the physical objects by displaying each frame ofthe live video feed. Further, displays 220 and 222 can be configured todisplay virtual objects by overlaying virtual vehicles onto eachdisplayed frame.

FIGS. 3A-D are diagrams showing how visible portions of virtual vehicle322 and visible portions of virtual vehicle 324 are displayed on adisplay 320, according to some embodiments. For ease of explanation,diagram 300 will be described with respect to the elements, (e.g.,display system 102, vehicle operator 114, and simulation system 140) ofFIG. 1. FIG. 3A shows an example real-world view 302A that may beobserved by vehicle operator 114 through conventional displays. FIG. 3Bshows an example virtual rendering 332 of real-world view 302A toinclude virtual vehicles 343 and 344. FIG. 3C shows an example ofdisplay 320 for displaying visible portions 322 and 324 of virtualvehicles 343 and 344, respectively. Display 320 may correspond to adisplay implemented within a visor, a helmet (e.g., helmet 116), orother headgear worn by an operator (e.g., vehicle operator 114) sittingwithin and driving a physical vehicle (e.g., physical vehicle 101). FIG.3D shows an example augmented view 302B that may be observed by vehicleoperator 114 through display 320.

As shown in FIG. 3A, real-world view 302A shows the field of view ofvehicle operator 114 if virtual vehicles were not displayed, i.e., viaconventional displays. As shown in real-world view 302A, vehicleoperator 114 may see, through display 320, other physical vehicles suchas physical vehicle 310A on a physical racecourse as well as physicalobjects within physical vehicle 101. For example, such physical objectsmay include, without limitation, rearview mirror 304A, vehicle frame306A, windshield wipers 308A, dashboard 312A, etc. In some embodiments,physical objects may include the hands or arms of vehicle operator 114.Additionally, as shown in real-world view 302A, vehicle operator 114 maysee a shadow of physical vehicle 310A. In some embodiments, vehicleoperator 114 may see physical vehicles and physical objects throughdisplay 320 because display 320 can be a transparent or semi-transparentdisplay.

As discussed above with respect to FIG. 1, kinematics information ofphysical vehicle 101 (e.g., position information) and a position of theoperator's point of view may be transmitted to simulation system 140configured to provide visible portions of virtual vehicles 313 and 314.In some embodiments, based on the kinematics information and theposition of the operator's point of view, simulation system 140 cancalculate a virtual world to include virtual vehicles and arepresentation of physical vehicle 101 racing against each other on avirtual racecourse corresponding to the physical racecourse seen byoperator 114. In some embodiments, simulation system 140 can calculaterepresentations of the various physical objects within the virtualworld.

In some embodiments, to enable simulation system 140 to track andcalculate representations of the hands or arms of vehicle operator 114,vehicle operator 114 can wear gloves that embed one or more sensors(e.g., accelerometers, position sensors, etc.) that transmitposition-related measurements to simulation system 140. Based on sensormeasurements, e.g., position or acceleration information, simulationsystem 140 can calculate corresponding representations of arms or hands(not shown) in the virtual world.

In some embodiments, to enable simulation system 140 to track andcalculate representations of the hands or arms of vehicle operator 114,one or more cameras can be mounted in the physical vehicle beingoperated by vehicle operator 114. The one or more cameras may trackpositions of the arms and hands based on markers embedded in ordisplayed on gloves or tracksuit worn by vehicle operator 114. Forexample, markers may include specific colors, patterns, materials, etc.In these embodiments, the one or more camera may transmit the capturedinformation to simulation system 140 that calculates the correspondingrepresentations of arms or hands (not shown) in the virtual world.

As shown in FIG. 3B, simulation system 140 may calculate, within thevirtual world, a virtual rendering 332 of real-world view 302A. Invirtual rendering 332, simulation system 140 can calculate arepresentation 340 of physical vehicle 310A and virtual vehicles 343 and344. Additionally, simulation system 140 can calculate representations334, 336, 338, and 342 of corresponding physical objects: rearviewmirror 304A, vehicle frame 306A, windshield wipers 308A, and dashboard312A. As shown in virtual rendering 332, simulation system 140 canexclude calculating representations of physical objects that do notobstruct view of virtual vehicles 343 and 344. For example, thespeedometer and steering wheel as seen by vehicle operator 114 inreal-world view 302A may not be calculated by simulation system 140 invirtual rending 332. In some embodiments, as shown in virtual rendering332, simulation system 140 can calculate shadows of physical vehicle 340and virtual vehicles 343 and 344.

In some embodiments, simulation system 140 can calculate portions ofvirtual vehicles 322 and 324 to display on display 320 of FIG. 3C toenable vehicle operator 114 to compete against virtual drivers in thereal world. In some embodiments, a visible portion of a virtual vehiclefrom the position of the point of view of vehicle operator 114 is thatportion of the virtual vehicle that is not obstructed by objects in thevirtual world from a virtual position in the virtual world correspondingto the position of the point of view in physical vehicle 101. In someembodiments, the simulation system can convert the position of the pointof view of vehicle operator 114 to virtual coordinates within thevirtual world. For example, from the point of view of an operator of thephysical vehicle, the corresponding virtual position in the virtualworld would be inside a representation of physical vehicle 101 in thevirtual world. From the corresponding virtual position of the point ofview of physical operator 114 in the virtual world, a view of thevirtual vehicle may be obstructed by the simulated physical vehicle (forexample, representations of vehicle frame 306A or windshield wipers308A), other simulated physical vehicles (e.g., a representation ofphysical vehicle 310A), shadows, simulated trees and other stationaryobjects, the simulated racecourse (e.g., when the virtual vehicle is ina dip and partially obstructed by the course itself, etc.) The visibleportion of the virtual vehicle is then the unobstructed view of thevirtual vehicle. Further embodiments are described with respect to FIGS.6 and 9.

For example, as shown in virtual rendering 332 in FIG. 3B, the simulatedview of virtual vehicle 343 shows that portions of virtual vehicle 343are obstructed by a representation 336 of vehicle frame 306A. Similarly,in virtual rendering 332, the simulated view of virtual vehicle 343shows that portions of virtual vehicle 344 are obstructed by arepresentation 340 of physical vehicle 310A and a representation 338 ofwindshield wiper 308A.

In some embodiments, simulation system 140 may calculate visibleportions of a virtual vehicle 322 to be portions of virtual vehicle 343in virtual rendering 332 that are not obscured by a representation ofcar frame 336 in the virtual world. Similarly, simulation system 140 maycalculate visible portions of a virtual vehicle 324 to be portions ofvirtual vehicle 344 in virtual rendering 332 that are not obscured byrepresentations 340 and 338 of physical vehicle 310A and windshieldwiper 308A, respectively. In some embodiments, information related tothese calculated visible portions 322 and 324 can be transmitted tocomponents within physical vehicle 101 and displayed by display 320.

In some embodiments, as shown in FIG. 3C, components within physicalvehicle 101, such as display system 102, can display visible portions322 and 324 of virtual vehicles 343 and 344, respectively, on display320. In some embodiments, visible portions 322 and 324 can includeshadows of virtual vehicles 343 and 344 as calculated and shown invirtual rendering 332 of FIG. 3B. In some embodiments, by augmentingreal-world view 302A with visible portions 322 and 324 being displayedon display 320, display system 102 enables vehicle operator 114 to seeboth physical vehicles in the real world and virtual vehicles.

In some embodiments, augmented view 302B in FIG. 3D shows the field ofview of vehicle operator 114 once display 320 displays (e.g., asrendered by display system 102) visible portions of virtual vehicles 322and 324. For example, similar to view 302A, vehicle operator 114 maystill see, via display 320, various physical objects on the racecoursein the real world. For example, as shown in augmented view 302B, vehicleoperator 114 may still see rearview mirror 304B, vehicle frame 306B,windshield wiper 308B, physical vehicle 310B, and dashboard 312B.Additionally, vehicle operator 114 may see virtual vehicles 313 and 314being displayed. In some embodiments, virtual vehicles 313 and 314 seenby vehicle operator 114 respectively correspond to visible portions ofvirtual vehicles 322 and 324 being displayed on display 320, asdescribed above. In some embodiments, the techniques described withrespect to FIGS. 2A-C can be combined with the techniques described withrespect to FIGS. 3A-D.

FIGS. 4A-D are diagrams showing how visible portions of virtual vehicle422 are displayed on a display 420, according to some embodiments. FIG.4A shows an example real-world view 402A that may be observed by aspectator or imaged by a video camera. FIG. 4B shows an example virtualrendering 430 of real-world view 402A to include virtual vehicle 434.FIG. 4C shows an example of display 420 for displaying visible portions422 of virtual vehicle 434. Display 420 may correspond to a displayimplemented within a visor, a helmet (e.g., helmet 116), or otherheadgear worn by a viewer (e.g., an audience member) present at aphysical racecourse. FIG. 4D shows an example augmented view 402B thatmay be observed by an audience member or imaged by a video camerathrough display 420.

As shown in FIG. 4A, real-world view 402A shows the field of view of theviewer if virtual vehicles were not displayed, i.e., via conventionaldisplays. As shown in real-world view 402A, the viewer may see, viadisplay 420, other viewers 404A and various physical objects in the realworld. For example, such physical objects may include fence 406A, publicannouncement (PA) horn speakers 408A, and physical vehicles 410A and412A, and shadows 411A and 413A of respective physical vehicles 410A and412A. In some embodiments, the viewer may see physical vehicles andphysical objects through display 420 because display 420 can be atransparent or semi-transparent display.

In some embodiments, in addition to fixed objects like fences or walls,the physical objects discussed above that may obstruct the viewer'sfield of view of the racecourse can include non-stationary objects whosepositions may change over time. For example, such non-stationary objectsmay include the heads of audience members or the bodies of audiencemembers when they stand. In some embodiments, to enable simulationsystem 140 to accurately calculate representations of both fixed andnon-stationary objects, the viewer's headset can include a camera facingthe racecourse and capturing a portion of the racecourse. The lines andborders of the racecourse or other markers placed on the racecourse canbe detected by the camera to determine whether one or more physicalobjects are obstructing the viewer's view of the racecourse. Forexample, the camera can detect omitted portions or breaks in the edges,lines, or markers of the racecourse. In some embodiments, informationabout the breaks or omitted portions can be transmitted to simulationsystem 140. In some embodiments, simulation system 140 can determinewhich portions of the virtual vehicle are obstructed by physical objectsby determining an overlapping portion of the virtual vehicle with theone or more breaks. Since a break indicates that the viewer's view isbeing blocked by a physical object, the simulation system 140 can setthe alpha values of pixels in the overlapping portion to “zero percent”(in RGBA) to make these overlapping portions transparent.

In some embodiments, information related to the viewer's point of viewmay be transmitted to a simulation system (e.g., simulation system 140or simulation system 540 of FIG. 5) configured to provide visibleportions of virtual vehicle 422 to the viewer. For example, suchinformation may include a position of the viewer's point of view. Asdescribed with respect to FIG. 6, the simulation system can calculate avirtual world to include virtual vehicles and representations ofphysical vehicles racing against each other on a virtual racecourse. Insome embodiments, simulation system 140 can calculate representations ofthe various physical objects (e.g., PA speakers 408A) within the virtualworld.

As shown in FIG. 4B, the simulation system may calculate, within thevirtual world, a virtual rendering 430 of real-world view 402A. Invirtual rendering 430, the simulation system can calculate virtualvehicle 434 and representations 440 and 432 of physical vehicles 410Aand 412A. In some embodiments, virtual vehicle 434 as calculated by thesimulation system can include a calculated shadow 435. Similarly,representations 440 and 432 of physical vehicles 410A and 412A can becalculated to include respective shadows 441 and 433. In someembodiments, the simulation system calculates shadows 441 and 433 in thevirtual world to correspond to respective shadows 411A and 413A as seenby the viewer in real-world view 402A. Additionally, the simulationsystem can calculate representations 436 and 438 of correspondingphysical objects: fence 406A and PA speakers 408A. As shown in virtualrendering 430, the simulation system can exclude calculatingrepresentations of physical objects that do not obstruct view of virtualvehicle 422. For example, audience (e.g., viewers 404A) in real-worldview 402A may not be calculated by the simulation system in virtualrending 430.

In some embodiments, the simulation system can calculate portions ofvirtual vehicle 422 to display on display 420 of FIG. 4C to enable theviewer to see a race between physical vehicles 410A and 412A and virtualvehicles such as virtual vehicle 434 being simulated in the virtualworld. In some embodiments, a visible portion of a virtual vehicle fromthe position of the viewer's point of view is that portion of thevirtual vehicle that is not obstructed by objects in the virtual worldfrom a virtual position in the virtual world corresponding to theposition of the viewer's point of view. In some embodiments, thesimulation system can convert the position of the viewer's point of viewto virtual coordinates within the virtual world. From the correspondingvirtual position of the viewer's point of view in the virtual world, aview of the virtual vehicle may be obstructed by simulated physicalvehicles (for example, representations of physical vehicles 410A and412A) and other simulated objects (e.g., representations of fence 406Aand horn speakers 408A), simulated trees, the simulated racecourse(e.g., when the virtual vehicle is in a dip and partially obstructed bythe course itself, etc.). The visible portion of the virtual vehicle isthen the unobstructed view of the virtual vehicle. Further embodimentsare described with respect to FIGS. 6 and 9.

For example, as shown in virtual rendering 432 in FIG. 4B, the simulatedview of virtual vehicle 434 shows that portions of virtual vehicle 434are obstructed by a representation 438 of PA speaker 408A and arepresentation 432 of physical vehicle 412A.

In some embodiments, the simulation system may calculate visibleportions of virtual vehicle 422 to be portions of virtual vehicle 434that are not obscured by representation of PA horn speakers 438 invirtual rendering 430 and that are not obscured by representation ofphysical vehicle 432 in the virtual world. In some embodiments,information related to the calculated visible portions 422 can betransmitted to the viewer and displayed by display 420.

In some embodiments, as shown in FIG. 4C, components worn by the viewer,such as display system 592 of FIG. 5, can display visible portions 422of virtual vehicle 434 on display 420. As shown in FIG. 4C, visibleportions 422 can include car frame and details 424 and a shadow 426 ofvirtual vehicle 434. In some embodiments, visible portions 422 caninclude a shadow 428 being cast on virtual vehicle 434 as calculated invirtual rendering 430. For example, shadow 428 may be a shadow cast byrepresentation 432 of physical vehicle 412A being calculated in virtualrendering 430. In some embodiments, by augmenting real-world view 402Awith visible portion 422 being displayed on display 420, a displaysystem (e.g., display system 592) enables the viewer to see bothphysical vehicles in the real world and virtual vehicles.

In some embodiments, augmented view 402B in FIG. 4D shows the field ofview of the viewer once display 420 displays (e.g., as rendered bydisplay system 592) visible portions of virtual vehicle 422. Forexample, similar to view 402A, the viewer may still see, via display420, various physical objects on the racecourse in the real world. Forexample, as shown in augmented view 402B, the viewer may still see otherviewers 404B, fence 406B, public announcement (PA) horn speakers 408B,and physical vehicles 410B and 412B as well as their respective shadows411B and 413B. Additionally, the viewer may see virtual vehicle 414 andshadow 415 of virtual vehicle 414 being displayed. In some embodiments,virtual vehicle 414 seen by the viewer can correspond to visibleportions of virtual vehicle 422 being displayed on display 420, asdescribed above. In the example of augmented view 402B, virtual vehicle414 seen by the viewer is obstructed by PA horn speakers 408B andphysical vehicle 412B. In some embodiments, virtual vehicle 414 beingdisplayed can overlap portions of shadow 413A as may be seen by theviewer in real-world view 402A. As a result and as shown by shadow 413Bin augmented view 402B, portions of shadow 413A may be obscured byvisible portions of virtual vehicle 422. In some embodiments, visibleportions of virtual vehicle 422 can include shadow 428 of physicalvehicle 412A. In these embodiments and as shown in augmented view 402B,the viewer can see shadow 416 being cast on virtual vehicle 414.

In some embodiments, augmented view 402B can be provided to viewer via anon-transparent display based on a full rendering technique as describedwith respect to FIGS. 2A-C. In these embodiments, a camera coupled to aviewer's headset can capture one or more video frames of the viewer'sfield of view as seen in real-world view 402A. The simulation system cansimilarly calculate virtual vehicle 434 within virtual rendering 430.However, in these embodiments, instead of displaying only visibleportions of virtual vehicle 422 on display 420, a display implemented inthe viewer's headset can be configured to output the one or morecaptured video frames with visible portions of virtual vehicle 422overlaid thereon. In some embodiments, physical objects (e.g., fences,beams, infrastructure, etc.) may block the viewer's view of theracecourse. Because augmented view 402B may be fully rendered views, thedisplay may overlay both visible vehicles and representations ofphysical vehicles (as calculated by the simulation system) on top of thecaptured video frames such that large physical objects do not block theviewer's view of the race for extended periods of time.

In some embodiments, as will be further described with respect to FIGS.5, 6, and 9, real-world view 402A may correspond to a live video feedcaptured by a camera (e.g., video camera 580 of FIG. 5) installed at aphysical racecourse. In these embodiments, information related to apoint of view of the camera may be transmitted to the simulation system.For example, such information may include one or more of the camera'sposition, orientation, tilt, or degree of rotation. Based on theinformation of the camera's point of view, the simulation system maycalculate a virtual world to include virtual rendering 430 to includevirtual vehicle, representations of physical vehicles, andrepresentations of physical objects, as described with respect to FIG.4B. In some embodiments, based on virtual rendering 430, the simulationsystem can calculate visible portions of virtual vehicle 422 as would beviewable at the position of the camera's point of view. As describedwith respect to FIG. 4C, visible portions of virtual vehicle 422 mayinclude car frame and details 424, shadow 426 of the virtual vehicle, orshadow 428 being cast by other objects in virtual rendering 430. In someembodiments, visible portions of virtual vehicle 422 may be overlaid onthe live video feed of real-world view 402A to display a race betweenphysical vehicles and virtual vehicles, as shown in augmented view 402Band as described above with respect to FIG. 4D.

FIG. 5 is a system 500 for simulating a virtual race between a physicalvehicle 510 and a simulated entity 530, according to some embodiments.In some embodiments, a network 502 communicatively couples variouscomponents: physical vehicle 510, simulated entity 530, simulationsystem 540, viewing devices 560, content-delivery system 570, camera580, and viewer 590. In some embodiments, camera 580 can be coupled tocontent-delivery system 570. As shown in system 500, network 502 may bea conduit for data flows between the various components. Network 502 canbe a wired and/or wireless network that includes any combination oflocal area networks (LANs), wide area networks (WANs), the Internet, andthe like.

In some embodiments, simulation system 540 include a number of engines552 that operate on models 542 and information received from physicalvehicle 510 and simulated entity 530 to simulate the virtual race,represented as racing simulation 550. Further, simulation system 540includes RF circuitry 548, which may include similar components as RFcircuitry 105 described with respect to FIG. 1, for communicating data(e.g., graphics data, kinematics information, force information, audioinformation, etc.) with physical vehicle 510, viewer 590, and simulatedentity 530.

In some embodiments, models 542 include vehicle models 544 andracecourse models 546. Vehicle models 544 may include 3-D models ofanimate or inanimate objects in the virtual environment of racingsimulation 550. For example, vehicle models 544 may include a 3-D modelof physical vehicle 510 as well as a 3-D model of a virtual vehiclecorresponding to simulated entity 530. Racecourse models 546 may include2-D or 3-D models of the physical racecourse on which physical vehicle510 is operating. In some embodiment, the 2-D or 3-D models may includeinformation related to terrains, boundaries, or topological features andthe like. Racecourse models 546 may include a racecourse and relatedcharacteristics (e.g., terrain, material type, length, etc.), stands foran audience, etc.

In some embodiments, to generate and maintain racing simulation 550,engines 552 include a physics engine 554, a graphics engine 556, and anaudio engine 558. Physics engine 554 may include algorithms foremulating the laws of physics realistically within racing simulation550. In particular, physics engine 554 includes algorithms to controlhow components, e.g., simulated physical vehicles or simulated virtualvehicles, interact with each other and a virtual racecourse in racingsimulation 550. In some embodiments, as described elsewhere herein,physics engine 554 generates and maintains racing simulation 550 basedon kinematics information received from physical vehicle 510 and basedon inputs received from simulated entity 530. As will be furtherdescribed with respect to FIGS. 6 and 9, the kinematics information caninclude position information of physical vehicle 510, according to someembodiments. For example, physics engine 554 may generate an avatar ofphysical vehicle 510 within racing simulation 550 based on acorresponding model in vehicle models 544 where a position of the avatarin racing simulation 550 may be calculated based on the receivedkinematics information. Additionally, physics engine 554 may generatekinematics information of a virtual vehicle corresponding to simulatedentity 530 based on the inputs received from simulated entity 530. Usingthe generated kinematics information and vehicle models 544, physicsengine 554 may simulate the virtual vehicle on the virtual racecoursewithin racing simulation 550.

In some embodiments, to enable physical vehicle 510 to simulate one ormore virtual vehicles on display 512, simulation system 540 transmitsthe kinematics information of the virtual vehicle to simulationcomponent 522 or display system 514.

In some embodiments, as described with respect to simulation system 140of FIG. 1, physics engine 554 (within simulation system 540) furthercalculates force information based on an interaction simulated betweenthe physical vehicle and the virtual vehicle in racing simulation 550.In some embodiments, physics engine 554 calculates force information forphysical vehicle 510 and force information for simulated entity 530.Then, simulation system 540 may transmit the calculated forceinformation to physical vehicle 510, simulated entity 530, or both viaRF circuitry 548.

In some embodiments, to enhance the realism of the race experienced atphysical vehicle 510 and simulated entity 530, audio engine 558 (withinsimulation system) include algorithms for calculating sounds withinracing simulation 550. Audio engine 558 may include sound files relatedto engines, brakes, tires, explosions, as well as collisions betweenvehicles. In some embodiments, audio engine 558 calculates a volume of agenerated sound based on a distance between vehicles as calculated byphysics engine 554 to generate racing simulation 550. Then, audio engine558 may transmit audio information to physical vehicle 510, simulatedentity 530, or both.

In some embodiments, graphics engine 556 generates 3-D animated graphicsfor racing simulation 550. For example, graphics engine 556 may utilizespecialized hardware for rendering vehicles (e.g., an avatar of physicalvehicle 510 or a virtual vehicle corresponding to simulated entity 530)based on vehicle models 544 and computations from physics engine 554.Further, graphics engine 556 may render a virtual racecourse withinracing simulation 550 based on racecourse models 546. In someembodiments, the graphics information (e.g., vehicles and racecourse)generated by graphics engine 556 can be transmitted to physical vehicle510, simulated entity 530, or a combination thereof via RF circuitry548.

Graphics engine 556 may utilize techniques such as rasterization orray-tracing for generating the 3-D animated graphics. In someembodiments, graphics engine 556 includes computer software applicationsthat are programmed and compiled to be executed on one or moreprocessors of simulation system 540. In other embodiments, graphicsengine 556 can be built upon graphics application programming interfaces(APIs) such as Direct3D or Open GL.

In some embodiments, physical vehicle 510 corresponds to physicalvehicle 101 described with respect to FIG. 1. To simulate a virtualvehicle, corresponding to simulated entity 530, at physical vehicle 510,physical vehicle 510 includes one or more of the following components:display 512, display system 514, telemetry system 520 (which may includesensors 516), force controller 518, and simulation component 522. Thesecomponents may correspond to the similarly named components describedwith respect to FIG. 1. In general, telemetry system 520 can be two-waytelemetry systems that receive and transmit data. For example, telemetrysystem 520 may transmit data, e.g., kinematics information of physicalvehicle 510, monitored by sensors 516 to simulation system 540 vianetwork 502. In some embodiments, the data includes a position ofphysical vehicle 510 on a physical racecourse. In some embodiments, thedata captured by sensors 516 may include a position of a point of viewof an operator of physical vehicle 510, as will be further describedwith respect to FIGS. 6 and 9.

In some embodiments, telemetry system 520 can receive kinematicsinformation of the virtual vehicle from simulation system 540 asdescribed above. Telemetry system 520 may route the received kinematicsinformation to display system 514. Based on the received kinematicsinformation, display system 514 may generate a virtual representation ofthe virtual vehicle that is processed by display system 514 for displayon display 512. By simulating the virtual vehicle within a field of viewof the operator of physical vehicle 510, system 500 enables the operatorto feel as if the simulated vehicle is in physical proximity to theoperator. As described above, the processing for generating the virtualrepresentation of the virtual vehicle may be performed remotely, e.g.,offloaded to simulation system 540. In these embodiments, the virtualrepresentation can be generated and transmitted by simulation system 540to display system 514. Display system 514 may then display the virtualrepresentation on display 512 to simulate the virtual vehicle within thefield of view of the operator.

In some embodiments, as will be further described with respect to FIGS.6 and 9, to enhance the realism of the virtual representation, thegenerated virtual representation can include a portion of the virtualvehicle that is visible from the position of the point of view of theoperator. In some embodiments, simulation system 540 can generate thevisible portion. In some embodiments, simulation component 522 onboardphysical vehicle 510 can generate the visible portion.

In some embodiments, to further enhance the realism of the race asexperienced by the operator of physical vehicle 510, force controller518 receives force information to control one or more mechanicalelements in physical vehicle 510. In some embodiments, as describedabove, the force information as calculated by physics engine 554 isreceived from simulation system 540. In other embodiments, the forcecalculation may be performed on-board physical vehicle 510.

In some embodiments, viewer 590 may correspond to an audience memberwatching the race between physical vehicle 510 and simulated entity 530.To simulate a virtual vehicle for display to viewer 590, viewer 590 maywear or operate one or more devices that implement one or more of thefollowing components: display 594, display system 592, telemetry system596 (which may include sensors 598), and simulation component 599. Thesecomponents may correspond to the similarly named components describedwith respect to physical vehicle 510. In general, telemetry system 596transmits limited kinematics information of viewer 590 as detected bysensors 598. For example, the limited kinematics information may includea position (e.g., GPS position) of viewer 590. As viewer 590 is likelyassociated with limited motion, other types of kinematics informationrelated to motion may not need to be captured by sensors 598, accordingto some embodiments. In some embodiments, like sensors 516, sensors 598may include cameras for capturing a position of a point of view ofviewer 590. In some embodiments, display 594 can be implemented in oneor more devices worn by viewer 590. For example, display 594 may beimplemented in a headset (e.g., a helmet) or a visor worn by viewer 590.In some embodiments, viewer 590 may wear or operate a device thatimplements simulation component 599. Similar to the functionality ofsimulation component 522, simulation component 599 may process some ofthe computations performed by simulation system 540.

In some embodiments, simulation system 540 communicates withcontent-delivery system 570 to display the competition between live andsimulated participants, e.g., physical vehicles and virtual vehicles, toan audience via viewing devices 560. For example, the virtual vehiclemay be overlaid onto a real video footage from the perspective of avideo camera 580 at the physical racecourse, and the combined videofootage may be shown on one or more viewing devices 560 such that theaudience which would see the physical vehicle and virtual vehicle on thesame racecourse in competition.

In some embodiments, content-delivery system 570 includes: video server572 for broadcasting video content via a cable or television network;and web server 574 for transmitting video content on-demand or via livestreaming via network 502, e.g., the Internet. As discussed above, videoserver 572 can broadcast video content obtained via video camera 580. Insome embodiments, a plurality of video cameras may be present at thephysical racecourse to record live video footage of the race fromdifferent points of views. In these embodiments, video server 572 canselect live video footage captured by one video camera (e.g., videocamera 580) from the plurality of video cameras. In some embodiments,each video camera comprises its own point of view, and each point ofview is used to determine the visible portion of the virtual vehicle forcombining with the live image feed from each video camera.

Though video server 572 and web server 574 are shown as beingimplemented by content-delivery system 570, one or more of these servers(e.g., video server 572 and web server 574) may be implemented byseparate entities or by simulation system 540.

In some embodiments, viewing devices 560 include a variety of electronicdevices with displays for presenting video data. For example, audiencesattending a live event/competition may watch the competition on viewingdevices 560 that include television (TV) screens, jumbotrons, or thelike. In another example, audiences at home may operate viewing devices560 such as TVs, laptops, tablets, smartphones, desktop computers, amongother types of mobile devices. Further, in some embodiments, audiencesboth at home and at the live competition may wear viewing devices 560such as HMDs or goggles that would display the combined scene (e.g.,including live and virtual participants) from the perspective of theaudience member based on location-based information of where theaudience member is located, as well as head and eye spatial anddirectional measurements to accurately recreate the scene usingprocessors built into the HMDs, on portable computers or mobile deviceswith the audience, or from remote servers with camera and audiencemember positional locations registered in them and which stream thedisplay information to the audience member's HMD or portablecomputer/mobile device.

In some embodiments, simulated entity 530 includes a simulation device532 coupled to a display 534 and an input controller 536 for controllingthe virtual vehicle in a virtual race. In some embodiments, simulationdevice 532 includes force controller 538 and display system 539.Simulation device 532 may be a general-purpose computer or aspecial-purpose computer such as a videogame console.

In some embodiments, input controller 536 may include a keyboard, avideogame controller, a joystick, a steering wheel and pedals, a forcepad, a treadmill, a steering wheel, among other types of input devicesfor controlling the virtual vehicle in a virtual race.

In some embodiments, simulation device 532 receives inputs from theinput controller 536 and transmits the received inputs to simulationsystem 540. In some embodiments, simulation system 540 simulates thevirtual vehicle on the virtual racecourse based on the inputs. Then,simulation system 540 may transmit display information corresponding toa position of the virtual vehicle on the virtual racecourse. Displaysystem 539 within simulation device 532 may receive the displayinformation and render the virtual race as computer-generated imagery(CGI) on display 534. In some embodiments, display system 539 projectsthe virtual race on display 534.

As shown in system 500, display 534 may include a television screen, amonitor, a projector, or other devices for displaying graphics data. Insome embodiments, as described with respect to displays of FIG. 1,display 534 may include a head-mounted display (HMD) worn by a user ofinput controller 536. For example, the HMD may be include a visor, aheadset (e.g., a helmet), glasses, goggles, or other devices worn infront of the user's eyes.

In other embodiments, the HMD implements retinal projection techniquesto simulate one or more virtual vehicles. For examples, the HMD mayinclude a virtual retinal display (VRD) that projects images onto theleft and right eyes of vehicle operator 114 to create athree-dimensional (3D) image of one or more virtual vehicles in thefield of view of vehicle operator 114.

In some embodiments, force controller 538 receives force informationfrom simulation system 540. The force information may be associated withan interaction, e.g., a collision, simulated by simulation system 540between the virtual vehicle and the physical vehicle on the virtualracecourse. To enhance the virtual racing experience for simulatedentity, force controller 538 may provide feedback to input controller536 by, for example, by vibrating input controller 536. In someembodiments, a user operating input controller 536 may wear a hapticsuit including one or more actuators controlled by force controller 538to emulate the physical sensations that would be felt by the user in areal collision.

FIG. 6 is a flowchart illustrating a method 600 for displaying virtualvehicles on displays, according to some embodiments. In someembodiments, method 600 includes steps performed at a physical vehicle602, a simulation system 604, and a viewer 606. For example, stepsperformed at physical vehicle 602 may be implemented by componentswithin a physical vehicle such as physical vehicle 101 of FIG. 1 orphysical vehicle 510 of FIG. 5. For example, steps performed atsimulation system 604 may be implemented by simulation system 140 ofFIG. 1 or simulation system 540 of FIG. 5. For example, steps performedat viewer 606 may be performed by devices (e.g., components shown inviewer 590 of FIG. 5) worn by an audience member watching a race betweenphysical vehicle 602 and a virtual vehicle on a physical racecourse.

In step 610, physical vehicle 602 identifies a position of physicalvehicle 602. In some embodiments, physical vehicle 602 can identify theposition of physical vehicle 602 by detecting kinematics information ofphysical vehicle 602 via one or more sensors (e.g., sensors 108 of FIG.1 or sensors 516 of FIG. 5) on board physical vehicle 602. In someembodiments, the position of physical vehicle 602 includes locationinformation for each of two portions of physical vehicle 602. Forexample, the location information for a first portion of physicalvehicle 602 may be detected by a GPS sensor placed at the first portion.In some embodiments, the position of physical vehicle 602 includes alocation of one portion of physical vehicle 602 and an orientation ofphysical vehicle 602. In some embodiments, the orientation of physicalvehicle 602 can include gyroscope data detected by a sensor (e.g., agyroscope) on-board physical vehicle 602.

In step 611, physical vehicle 602 provides the position of physicalvehicle 602 to simulation system 604.

In step 620, simulation system 604 receives inputs controlling a virtualvehicle. In some embodiments, the inputs can be received from inputcontroller 536 as described with respect to FIG. 5.

In step 621, simulation system 604 calculates a virtual world forsimulating a race on a virtual racecourse between the virtual vehicleand the physical vehicle based on inputs from physical vehicle 602 andinputs controlling the virtual vehicle. In some embodiments, the virtualworld can be implemented in a racing simulation (e.g., racing simulation141 of FIG. 1 or racing simulation 550 of FIG. 5). In some embodiments,the inputs from physical vehicle 602 can include the position ofphysical vehicle 602 provided in step 611. In some embodiments, togenerate the racing simulation, simulation system 604 can calculate arepresentation of the physical vehicle to add to the virtual racecoursein the racing simulation based, in part, on the position provided instep 611. In some embodiments, to generate the racing simulation,simulation system 604 can calculate the virtual vehicle to add to thevirtual racecourse in the racing simulation based on the inputs of step620. In some embodiments, simulation system 604 can use the inputs toupdate kinematics information associated with the virtual vehicle in thevirtual world. In some embodiments, the kinematics information includesone or more vectors of motion, one or more scalars of motion, a positionvector, a GPS location, a velocity, an acceleration, an orientation, ora combination thereof of the virtual vehicle. Based on the updatedkinematics information, simulation system 604 can update a simulation ofthe virtual vehicle in the virtual world, according to some embodiments.

In step 612, physical vehicle 602 identifies a position of a point ofview of an operator of physical vehicle 602. In some embodiments, theposition of the point of view of the operator can be determined withrespect to a head of the operator as detected by a sensor (e.g., acamera) in physical vehicle 602. For example, the position of the pointof view can include detecting a spatial position of the operator's head.For example, in some embodiments, the position of the operator's pointof view can be determined by at least one of the following: trackinghead position, identifying a vector from a point on the head to a fixedpoint on the physical vehicle, identifying a vector from a point on ahead gear to a fixed point on the physical vehicle, identifying a vectorfrom a point on the head to a fixed point in a venue, or identifying avector from a point on a head gear to a fixed point in the venue. Insome embodiments, the venue may include the physical racecourse, thestands, or other infrastructure at the physical racecourse.

In some embodiments, the position of the point of view of the operatorcan be determined with respect to eyes of the operator as detected by asensor (e.g., a camera) in physical vehicle 602. For example, theposition of the point of view can include detecting a spatial positionof a user's eyes, a gaze direction of the user's eyes, or a focus pointof the user's eyes where the user is the operator. For example, in someembodiments, the position of the operator's point of view can bedetermined by at least one of the following: measuring a point of gazeof eyes, tracking eye movement, identifying a vector from one or botheyes to a fixed point on the physical vehicle, identifying a vector froma point on eye-wear (e.g., a visor) to a fixed point in a venue,identifying a vector from one or both eyes to a fixed point on theracecourse, or identifying a vector from one or both eyes to a fixedpoint in the venue. In some embodiments, the venue may include thephysical racecourse, the stands, or other infrastructure at the physicalracecourse. In some embodiments, the position of the point of view ofthe operator can be identified by measuring light reflection orrefraction from the eyes.

In step 613, physical vehicle 602 provides a position of the point ofview of the operator to simulation system 604. In some embodiments,physical vehicle 602 wirelessly transmits the position of the point ofview to simulation system 604. In some embodiments, a telemetry system(e.g., telemetry system 104 of FIG. 1) coupled to physical vehicle 602can perform the transmitting.

In step 622, simulation system 604 calculates a first portion of thevirtual vehicle visible from the position of the point of view of theoperator. In some embodiments, as will be further described with respectto FIG. 9, simulation system 604 calculates the first portion bydetermining which portions of the virtual vehicle are unobscured by therepresentation of the physical vehicle (with respect to a virtualposition within the racing simulation generated in step 621). Inparticular, the simulation system 604 can determine the virtual positionto correspond to the position of the point of view provided by physicalvehicle 602 as described in step 613. In some embodiments, thesimulation system 604 calculates the first portion by determining whichportions of the virtual vehicle are unobscured by the representation ofphysical objects in the racing simulation. In some embodiments, asdescribed above with respect to FIGS. 3A-D, portions of the virtualvehicle within the racing simulation may be obstructed byrepresentations of other physical or virtual vehicles with respect tothe virtual position of the point of view of the operator. For example,a representation of a physical object may obscure a portion of thevirtual vehicle from a virtual position of the point of view when therepresentation of the physical object is positioned on a straight linebetween the virtual position of the point of view and the obscuredportion of the representation of the physical object. In someembodiments, the first portion calculated by simulation system 604 caninclude the unobstructed portion of the virtual vehicle, as describedabove. In some embodiments, the first portion can exclude the obstructedportions of the virtual vehicle, as described above.

In some embodiments, the first portion of the virtual vehicle caninclude one or more virtual shadows being generated within the virtualworld. In some embodiments, the one or more virtual shadows can includea virtual shadow of the virtual vehicle, a virtual shadow being cast onthe virtual vehicle, or both the virtual shadow of the virtual vehicleand the virtual shadow being cast on the virtual vehicle. For example,the virtual shadow being cast on the virtual vehicle may include avirtual shadow of another virtual vehicle, a virtual shadow of arepresentation of the physical vehicle in the virtual world, or avirtual shadow of other virtual objects being generated in the virtualworld.

In some embodiments, the first portion of the virtual vehicle caninclude a virtual representation as generated by simulation system 604(e.g., graphics engine 556). In some embodiments, the virtualrepresentation includes a set of graphical elements. In someembodiments, the virtual representation can be generated by simulationsystem 604 based on a digital 3-D model of the virtual vehicle stored ina database of models (e.g., vehicle models 544).

In step 623, simulation system 604 outputs the first portion calculatedin step 622 to physical vehicle 602. In some embodiments, simulationsystem 604 wirelessly transmits (e.g., via RF circuitry 548 of FIG. 5)the first portion to physical vehicle 602.

In step 614, physical vehicle 602 provides the first portion to adisplay system (e.g., display system 102 of FIG. 1 or display system 514of FIG. 5). In some embodiments, the first portion received fromsimulation system 604 can include kinematics information calculated bysimulation system 604. In some embodiments, the first portion receivedfrom simulation system 604 can include graphical information. In someembodiments, the telemetry system (e.g., telemetry system 104 of FIG. 1)coupled to physical vehicle 602 can receive information related to thefirst portion of the virtual vehicle.

In step 615, physical vehicle 602 displays the first portion of thevirtual vehicle on a display (e.g., display 512 of FIG. 5) proximate tophysical vehicle 602. In some embodiments, the display system (e.g.,rendering component 107 of FIG. 1) renders the first portion of thevirtual vehicle on the display. In some embodiments, a renderingcomponent (e.g., rendering component 107 of FIG. 1) in the displaysystem translates the first portion of the virtual vehicle into avirtual representation for displaying on the display. In someembodiments, the virtual representation includes a set of graphicalelements. In some embodiments, the display system displays a series ofrepresentations of the virtual vehicle (each representation including avisible portion of the virtual vehicle output in step 623) over a periodof time by repeating one or more steps (e.g., steps 610-615 and 620-623)of method 600 to simulate a trajectory of the virtual vehicle on theracecourse in the field of view of the operator.

In some embodiments, the display system includes a simulation component(e.g., simulation component 106) that generates the virtualrepresentation. In some embodiments, the virtual representation isgenerated based on a digital 3-D model of the virtual vehicle. In someembodiments, the digital 3-D model is stored in memory of the displaysystem. The digital 3-D model may be received from, for example, thesimulation system.

In some embodiments, the display includes one or more windows ofphysical vehicle 602. In some embodiments, the display can include oneor more windows or mirrors of physical vehicle 602 such as any of thefollowing displays as described with respect to FIG. 1: front windshield120, rear-view mirror 122, rear windshield 124, side windows 126A and126B, side mirrors 128A and 128B. In some embodiments, the display canbe implemented within a visor or a headset (e.g., helmet 116 of FIG. 1)worn by the operator physical vehicle 602.

In some embodiments, simulation system 604 performs steps similar tosteps 622 and 623 to enable other viewers such as viewer 606 to view thevirtual vehicle from other points of view. In some embodiments, viewer606 can be an audience member at a live racing event and watching a racebetween physical vehicle 602 on a physical racecourse and the virtualvehicle not physical present on the physical racecourse.

Although illustrated together in one system, in some embodiments one ofthe first visible portion and second visible portion are calculated andoutput without calculating and output the other portion.

In step 630, a display system of viewer 606 receives a selection for asecond point of view. In some embodiments, the selection can be a pointof view of viewer 606. For example, viewer 606 may be an audience memberpresent at the physical racecourse and observing physical vehicle 602 onthe racecourse. In some embodiments, the selection can be a point ofview of a video camera present at the physical racecourse and imaging aportion of the racecourse on which physical vehicle 602 is racing. Whenphysical vehicle 602 is traveling on the portion of the racecourse beingcaptured by the video camera, the video camera may image physicalvehicle 602 on a video feed. When the physical vehicle is not travellingacross the portion of the racecourse being captured by the camera, thecamera may still capture the portion of the racecourse. In someembodiments, the selection for the second point of view can default tothe point of view of viewer 606 or a video camera.

In step 631, the display system of viewer 606 identifies a position ofthe second point of view. In some embodiments where the second point ofview is the point of view of viewer 606, the position of the secondpoint of view can be determined with respect to the head of viewer 606as detected by a sensor (e.g., sensors 598) proximate to viewer 606. Forexample, in some embodiments, the position of the second point of viewcan be determined by at least one of the following: tracking headposition of viewer 600, identifying a vector from a point on the head toa fixed point in a venue, identifying a vector from a point on a headgear to a fixed point in the venue. In some embodiments, the venue mayinclude the physical racecourse, the stands, or other infrastructure atthe physical racecourse.

In some embodiments where the second point of view is the point of viewof viewer 606, the position of the second point of view can bedetermined with respect to eyes of viewer 606 as detected by a sensor(e.g., sensors 598) proximate to viewer 606. For example, the positionof the second point of view can include detecting a spatial position ofa user's eyes, a gaze direction of the user's eyes, or a focus point ofthe user's eyes where the user is viewer 606. For example, in someembodiments, the position of the second point of view can be determinedby at least one of the following: measuring a point of gaze of eyes,tracking eye movement, identifying a vector from a point on eye-wear(e.g., a visor) to a fixed point in a venue, or identifying a vectorfrom one or both eyes to a fixed point in the venue. In someembodiments, the venue may include the physical racecourse, the stands,or other infrastructure at the physical racecourse. In some embodiments,the position of the second point of view can be identified by measuringlight reflection or refraction from the eyes of viewer 606.

In step 632, the display system of viewer 606 provides the position ofthe second point of view to simulation system 604.

In step 624, simulation system 604 calculates a second portion of thevirtual vehicle visible from the position of the second point of view.In some embodiments, as will be further described with respect to FIG.9, simulation system 604 calculates the second portion by determiningwhich portions of the virtual vehicle are unobscured by therepresentation of the physical vehicle with respect to a virtualposition within the racing simulation generated in step 621. Inparticular, the simulation system 604 can determine the virtual positionto correspond to the position of the second point of view provided bythe display system of viewer 606 as described in step 632.

In step 625, simulation system 604 outputs the second portion calculatedin step 624 to the display system of viewer 606. In some embodiments,simulation system 604 wirelessly transmits (e.g., via RF circuitry 548of FIG. 5) the second portion to the display system of viewer 606.

In step 633, a wireless interface proximate to viewer 606 provides thesecond portion to the display system of viewer 606. In some embodiments,the second portion received from simulation system 604 can includekinematics information calculated by simulation system 604. In someembodiments, the first portion received from simulation system 604 caninclude graphical information. In some embodiments where the displaysystem of viewer 606 includes the wireless interface, the display systemof viewer 606 can directly receive the second portion.

In step 634, the display system of viewer 606 the second portion of thevirtual vehicle on a display proximate to viewer 606. In someembodiments, the display system of viewer 606 renders the second portionof the virtual vehicle on the display. In some embodiments, the displayproximate to viewer 606 can be implemented in a visor or a helmet wornby viewer 606.

In some embodiments, a non-transitory computer readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display, the one or moreprograms including instructions for implementing any of the stepsdescribed above with respect to FIG. 6. In some embodiments, anon-transitory computer-readable storage medium comprisescomputer-readable instructions, which when executed by one or moreprocessors, causes the one or more processors to perform steps describedabove with respect to FIG. 6. In some embodiments, a system comprises atleast one of foregoing non-transitory computer readable storage mediums,and one or more processors configured to execute the instructions of thenon-transitory computer readable storage medium(s). In some embodiments,a device comprises one or more processors configured to perform any ofthe steps described above with respect to FIG. 6.

FIG. 7 is a flowchart illustrating a method 700 for providing two-wayinteractions between a virtual vehicle and a physical vehicle to anoperator of the physical vehicle, according to some embodiments. Method700 may, for example, be implemented by components within a physicalvehicle such as physical vehicle 101 of FIG. 1. In some embodiments,method 700 enhances method 600 to provide tactile and audio feedback inaddition to the visual feedback displayed and described with respect tomethod 600.

In step 702, a telemetry system (e.g., telemetry system 104) monitorskinematics information of the physical vehicle (e.g., physical vehicle101). In some embodiments the telemetry system includes one or moresensors to detect the kinematics information. For example, the one ormore sensors may include a GPS receiver, an accelerometer, aspeedometer, an orientation sensor, a gyroscope, among other types ofsensors.

In step 704, the telemetry system transmits the kinematics informationto a simulation system (e.g., simulation system 140). In someembodiments, the kinematics information is transmitted via RF circuitry(e.g., RF circuitry 105).

In some embodiments, the simulation system simulates a virtual racebetween the virtual vehicle and the physical vehicle in a virtual worldbased on the telemetered kinematics information. In some embodiments,the virtual world includes a virtual racecourse where the virtualvehicle and a representation of the physical vehicle are simulated onthe virtual racecourse. In some embodiments, the simulation systemcalculates a distance between the virtual vehicle and the physicalvehicle on the virtual racecourse. Based on the calculated distance, thesimulation system determines whether a contact (e.g., a collision)exists between the virtual vehicle and the physical vehicle on thevirtual racecourse. Then, the simulation system calculates forceinformation that corresponds to the determined contact.

In some embodiments, the simulation system calculates audio informationbased on the calculated distance and whether the contact exists. In someembodiments, the audio information includes one or more of the sounds ofengines, brakes, tires, explosions, or explosions as well as the volumelevel of the one or more sounds. For example, the simulation system maycalculate the volume of the one or more sounds to be inverselyproportional to the calculated distance between the virtual vehicle andthe physical vehicle on the virtual racecourse.

In step 706, a force controller (e.g., force controller 112) receivesforce information from the simulation system. For example, a displaysystem (e.g., display system 102) may receive and forward the forceinformation to the force controller. In some embodiments, some or all ofthe functionality of calculating the force information may be performedat the physical vehicle in a simulation component (e.g., simulationcomponent 106). In these embodiments, the simulation component receiveskinematics information of the virtual vehicle or other virtual objects,as will be described with respect to step 816 of FIG. 8. Then, thesimulation component may perform the force calculations to generate theforce information.

In step 708, the display component receives audio information from thesimulation system. In some embodiments, some or all of the functionalityof calculating the audio information may be performed at the physicalvehicle in the simulation component. In these embodiments, thesimulation component receives kinematics information of the virtualvehicle or other virtual objects, as will be described with respect tostep 816 of FIG. 8. Then, the simulation component may perform the audiocalculations to generate the audio information.

In step 710, the force controller controls one or more mechanicalelements implemented in the physical vehicle based on the received forceinformation. In some embodiments, the force controller transmitsinstructions to one or more force actuators (i.e., examples ofmechanical elements) to emulate the physical sensations that would befelt by the operator of the physical vehicle should there be realphysical contact between the physical vehicle and another vehicle,displayed as a virtual representation of the virtual vehicle. In someembodiments, the one or more force actuators may be implemented within aseat and head brace (e.g., seat and head brace 130) or within a hapticsuit (e.g., haptic suit 118) worn by the operator.

In some embodiments, the mechanical elements may include parts thataffect the functionality of the physical vehicle. For example, themechanical elements may include a steering wheel column, brakes,airbags, etc. Based on the received force information, the forcecontroller may, for example, lock the brakes, deploy the airbags,vibrate the steering wheel column, create a bumping force on a sectionof the vehicle, slow the car by reducing power, or control othermechanical and/or electrical elements within the physical vehicle.

In step 712, the display system can control one or more speakers of thephysical vehicle (e.g., speakers 132) to output the audio information.In some embodiments, the display system can control one or more speakerswithin a helmet worn by the operator (e.g., helmet 116) to output theaudio information.

In some embodiments, a non-transitory computer readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display, the one or moreprograms including instructions for implementing any of the stepsdescribed above with respect to FIG. 7. In some embodiments, anon-transitory computer-readable storage medium comprisescomputer-readable instructions, which when executed by one or moreprocessors, causes the one or more processors to perform steps describedabove with respect to FIG. 7. In some embodiments, a system comprises atleast one of foregoing non-transitory computer readable storage mediums,and one or more processors configured to execute the instructions of thenon-transitory computer readable storage medium(s). In some embodiments,a device comprises one or more processors configured to perform any ofthe steps described above with respect to FIG. 7.

FIG. 8 is a flowchart illustrating a method 800 for simulating a racebetween a virtual vehicle and a physical vehicle to provide two-wayinteractions, according to some embodiments. Method 800 may, forexample, be implemented by a simulation system such as simulation system140 described with respect to FIG. 1 or simulation system 540 describedwith respect to FIG. 5. As described with respect to FIGS. 1 and 5, thesimulation system simulates a virtual race between the virtual vehicleand the physical vehicle on a virtual racecourse in a virtual worldwhere the virtual racecourse is simulated to correspond to a physicalracecourse.

In step 802, the simulation system receives input from a controller(e.g., input controller 536) to control a virtual vehicle on the virtualracecourse. In some embodiments, the controller may be a keyboard, amouse, a video game controller, a joystick, a steering wheel and pedals,a gesture on a touch screen, or a combination thereof among other typesof input devices.

In step 804, the simulation system receives kinematics information forthe physical vehicle (e.g., physical vehicle 101 of FIG. 1). In someembodiments, the kinematics information is received from the physicalvehicle as described with respect to FIG. 1.

In step 806, the simulation system simulates the virtual race betweenthe virtual vehicle and the physical vehicle on the virtual racecourse.In some embodiments, the simulation system simulates the virtual raceaccording to one or more of steps 808-812. In step 808, the simulationsystem determines kinematics information of the virtual vehicle based onthe input received in step 802. For example, the input may include anamount of force applied to a videogame controller that is translated bythe simulation system into an acceleration amount. In step 810, thesimulation system determines an interaction between the virtual vehicleand the physical vehicle on the virtual racecourse by comparing thekinematics information between the virtual vehicle and the physicalvehicle. In some embodiments, the simulation system determines adistance between the virtual vehicle and the physical vehicle simulatedon the virtual racecourse to determine whether a contact (e.g., acollision) occurs. In step 812, the simulation system generates forceinformation based on the interaction determined in step 810. In step813, the simulation system generates audio information based on theinteraction determined in step 810.

In step 816, the simulation system transmits the kinematics informationof the virtual vehicle to the physical vehicle. In some embodiments, thephysical vehicle uses the kinematics information of the virtual vehicleto generate and display the virtual vehicle on a display of the physicalvehicle.

In step 818, the simulation system transmits the force information tothe physical vehicle as described with respect to step 706 of FIG. 7. Insome embodiments, the physical vehicle controls one or more mechanicalor electrical elements of the physical vehicle based on the forceinformation to emulate the physical sensations that would be felt by anoperator of the physical vehicle in a real collision.

In step 820, the simulation system transmits generated audio informationto the physical vehicle as described with respect to step 708 of FIG. 7.In some embodiments, the physical vehicle controls one or more speakersof the physical vehicle based on the audio information to emulate theauditory experience that would be felt by the operator of the physicalvehicle should the virtual vehicle be physically present on the physicalracecourse. For example, the one or more speakers may include vehiclespeakers or speakers implemented within a headset worn by the operator.

In step 814, the simulation system renders the virtual race on asimulation display (e.g., display 534 of FIG. 5).

In some embodiments, a non-transitory computer readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device, the one or more programs includinginstructions for implementing any of the steps described above withrespect to FIG. 8. In some embodiments, a non-transitorycomputer-readable storage medium comprises computer-readableinstructions, which when executed by one or more processors, causes theone or more processors to perform steps described above with respect toFIG. 8. In some embodiments, a system comprises at least one offoregoing non-transitory computer readable storage mediums, and one ormore processors configured to execute the instructions of thenon-transitory computer readable storage medium(s). In some embodiments,a device comprises one or more processors configured to perform any ofthe steps described above with respect to FIG. 8.

FIG. 9 is a flowchart illustrating a method 900 performed by asimulation system to enable display of virtual vehicles, according tosome embodiments. Method 900 may, for example, be implemented bysimulation system 140 described with respect to FIG. 1 or simulationsystem 540 described with respect to FIG. 5. In some embodiments, one ormore steps of method 900 may correspond to one or more steps performedby simulation system 604 as described with respect to FIG. 6.

In step 902, the simulation system receives inputs controlling a virtualvehicle. For example, the inputs may be received from input controller536 as described with respect to FIG. 5. In some embodiments, step 902corresponds to step 620 of FIG. 6.

In step 904, the simulation system receives a position of a physicalvehicle. In some embodiments, the position of the physical vehicle canbe provided by the physical vehicle, e.g., as described with respect tostep 611 of FIG. 6.

In step 906, the simulation system calculates a virtual world forsimulating a race on a virtual racecourse between the virtual vehicleand the physical vehicle. In some embodiments, the virtual world can bea racing simulation stored in racing simulation 550 as described in FIG.5. In some embodiments, step 906 corresponds to step 621 of FIG. 6. Insome embodiments, to calculate the virtual world, method 900 performssteps 908-912.

In step 908, the simulation system simulates the virtual vehicle on thevirtual racecourse in the virtual world based on the inputs received instep 902. In some embodiments, the inputs control kinematiccharacteristics that define how the virtual vehicle moves on the virtualracecourse in the virtual world.

In step 910, the simulation system calculates a representation of thephysical vehicle in the virtual world based on the position of thephysical vehicle received in step 904. In some embodiments, thesimulation system can simulate the race between the physical vehicle andthe virtual vehicle by adding the representation of the physical vehicleto the virtual world. In some embodiments, to calculate therepresentation of the physical vehicle, the simulation system transformsphysical coordinates associated with the position of the physicalvehicle to virtual coordinates within the virtual world.

In step 912, the simulation system calculates a plurality ofrepresentations of objects in the virtual world. In some embodiments, arepresentation of an object (from the plurality of representations)corresponds to a physical object that is present in a physicalracecourse being modeled in the virtual world. For example, the virtualracecourse in the virtual world may be simulated based on the physicalracecourse, which may include physical objects such as trees, banners,pit stops, etc. In some embodiments, a representation of an object (fromthe plurality of representations) corresponds to a virtual object thatis present on the virtual racecourse being simulated in the virtualworld and that is not present on the physical racecourse. For example,the virtual object may include, without limitation, simulated obstacles,smoke, walls, explosions, or debris resulting from a collision betweenthe virtual vehicle and the physical vehicle being simulated in thevirtual world.

In some embodiments, the virtual world simulated by the simulationsystem can include the plurality of calculated representations ofphysical objects. In some embodiments, the simulation system cancalculate the plurality of representations of physical objects byaccessing a database of representations.

In step 914, the simulation system receives a position of a point ofview at the racecourse. In some embodiments, the position can bereceived from a physical vehicle as described with respect to step 613of FIG. 6. In these embodiments, the position represents a position of apoint of view of an operator of the physical vehicle. In someembodiments, the position can be received from a viewer as describedwith respect to step 632 of FIG. 6. In these embodiments, the positionrepresents a position of a point of view selected by the viewer.

In step 916, the simulation system calculates a portion of the virtualvehicle visible from the position of the point of view received in step914. In some embodiments, step 916 corresponds to steps 622 or 621 asdescribed with respect to FIG. 6 based on a source of the position ofthe point of view received in step 914. In some embodiments, calculatingthe portion includes calculating a field of view from the virtualposition of the point of view. In these embodiments, the calculatedportion can be within the calculated field of view. In some embodiments,to calculate the portion of the virtual vehicle visible from the pointof view, method 900 performs steps 918-926.

In step 918, the simulation system calculates a virtual position of thepoint of view within the virtual world based on the position of thepoint of view received in step 914. In some embodiments, to calculatethe virtual position, the simulation system transforms the physicalcoordinates of the position of the point of view to virtual coordinateswithin the virtual world.

In step 920, the simulation system determines whether one or morerepresentations of objects exist between the virtual position and thevirtual vehicle in the virtual world. In some embodiments, the one ormore representations of physical objects are selected from the pluralityof representations of physical objects calculated in step 912. Asdescribed above with respect to step 912, the one or morerepresentations of objects can include virtual representations ofphysical objects that are present on the physical racecourse. In someembodiments, the one or more representations of objects can includevirtual objects that are simulated in the virtual world are not presenton the physical racecourse. In step 922, if one or more representationsof the objects exist, method 900 proceeds to step 926. Otherwise, method900 proceeds to step 924.

In step 924, the simulation system extracts portions of the virtualvehicle that are unobscured, from the virtual position, by therepresentation of the physical vehicle in the virtual world.

In step 926, the simulation system extracts portions of the virtualvehicle that are unobscured, from the virtual position, by therepresentation of the physical vehicle and the one or morerepresentations of objects determined in step 920.

In step 928, the simulation system provides a portion of the virtualvehicle visible from the virtual position of the point of view toinclude one or more of the extracted portions. In some embodiments, theportion being output includes only the extracted portions. In someembodiments, as discussed above, the simulation system can calculate thefield of view from the virtual position of the point of view. In theseembodiments, the portion calculated by the simulation system caninclude: a non-excluded portion representing parts of the portion thatare visible within the calculated field of view; and an excluded portionrepresenting parts of the portion that are excluded (i.e., not visible)within the calculated field of view. In some embodiments where the fieldof view is calculated by the simulation system, the simulation systemcan calculate the portion to include only the non-excluded portionrepresenting parts of the portion that are visible within the calculatedfield of view.

In some embodiments, the simulation system provides the portion of thevirtual vehicle to the source originating the position of the point ofview as described with respect to step 914. For example, step 928 maycorrespond to steps 623 or 625 as described with respect to FIG. 6depending on a source of the point of view as described with respect tostep 914. In this example, if the position of the point of view isreceived from the physical vehicle, the simulation system may providethe portion of the virtual vehicle to the physical vehicle as describedwith respect to step 623 of FIG. 6.

In some embodiments, the virtual world can be a racing simulation storedin racing simulation 550 as described in FIG. 5. In some embodiments,step 906 corresponds to step 621 of FIG. 6. In some embodiments, tocalculate the virtual world, method 600 performs steps 908-912.

In some embodiments, a non-transitory computer readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device, the one or more programs includinginstructions for implementing any of the steps described above withrespect to FIG. 9. In some embodiments, a non-transitorycomputer-readable storage medium comprises computer-readableinstructions, which when executed by one or more processors, causes theone or more processors to perform steps described above with respect toFIG. 9. In some embodiments, a system comprises at least one offoregoing non-transitory computer readable storage mediums, and one ormore processors configured to execute the instructions of thenon-transitory computer readable storage medium(s). In some embodiments,a device comprises one or more processors configured to perform any ofthe steps described above with respect to FIG. 9.

FIG. 10 is a flowchart illustrating a method 1000 to enable display ofvirtual vehicles, according to some embodiments. Method 1000 may, forexample, be implemented by components within a physical vehicle such asphysical vehicle 101 of FIG. 1 or physical vehicle 510 of FIG. 5. Insome embodiments, by simulating the virtual vehicle within a field ofview of an operator of the physical vehicle on a racecourse, method 1000enhances the realism of the interaction between the physical vehicle andthe virtual vehicle as experienced by the operator against the virtualvehicle.

In step 1002, a sensor in the physical vehicle (e.g., eye-positiondetector 110) detects eyes measurements of an operator (e.g., vehicleoperator 114) of the physical system (e.g., physical vehicle 101). Insome embodiments, the sensor estimates the eyes measurements of the eyesbased on detecting the operator's head or a device worn on the head ofthe operator (e.g., helmet 116, visor over eyes 117, or a head-mounteddisplay (HMD)). For example, the sensor may estimate the eyesmeasurements of the operator's eyes based on detecting a position and/oran orientation of the device worn on the operator's head.

In step 1004, a display system in the physical vehicle (e.g., renderingcomponent 107) identifies a position (e.g., position 208A from FIG. 2)of a physical object in the field of view of the operator. In someembodiments, the position corresponds to a location on a display (e.g.,display 220 from FIG. 2) in proximity to the physical system.

In step 1006, the display system receives kinematics information of thevirtual vehicle representing a competitor vehicle not physically on theracecourse. Further, the display system may receive information from asimulation system (e.g., simulation system 140) related to virtualobjects within racing simulation 550 or racing simulation 141 that arenot physically present on the racecourse. In some embodiments, asdescribed with respect to FIG. 1, the kinematics information may includeGPS coordinates, spatial position, orientation, velocity, acceleration,or a combination thereof associated with the virtual vehicle. In someembodiments, the kinematics information can be received from asimulation system (e.g., simulation system 140) that simulates a racebetween the physical vehicle and the virtual vehicle on a simulatedracecourse.

In step 1008, the display system generates a representation of thevirtual vehicle based on the position of the physical object identifiedin step 1004, the eyes measurements detected in step 1002, and thekinematics information received in step 1006. In some embodiment, thedisplay system includes a simulation component (e.g., simulationcomponent 106) that generates the representation. Further, inembodiments where the display system receives information for othervirtual objects as described in step 1006, the display system similarlygenerates graphical, representations for these virtual objects. Forexample, virtual objects may include a wall, debris from a virtual car,or objects on a virtual racecourse being simulated in racing simulation550. In some embodiments, the virtual representation is generated basedon a digital 3-D model of the virtual vehicle. In some embodiments, thedigital 3-D model is stored in memory of the display system. The digital3-D model may be received from, for example, the simulation system.

In step 1010, the display system (e.g., rendering component 107)displays the representation of the virtual vehicle on the display toalign with the physical object represented by the identified position ofstep 1004. In some embodiments, a rendering component (e.g., renderingcomponent 107 of FIG. 1) in the display system translates therepresentation into a set of graphical elements for displaying on thedisplay. In some embodiments, the display system displays a series ofrepresentations of the virtual vehicle over a period of time byrepeating one or more steps (e.g., steps 1002-1010) of method 1000 tosimulate a trajectory of the virtual vehicle on the racecourse in thefield of view of the operator.

In some embodiments, the representation can be generated remotely by,for example, the simulation system. In these embodiments, the displaysystem receives information related to the representation as generatedby the simulation system. Further, the rendering component may translatethis received information into a set of graphical elements fordisplaying on the display.

In some embodiments, a non-transitory computer readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display, the one or moreprograms including instructions for implementing any of the stepsdescribed above with respect to FIG. 10. In some embodiments, anon-transitory computer-readable storage medium comprisescomputer-readable instructions, which when executed by one or moreprocessors, causes the one or more processors to perform steps describedabove with respect to FIG. 10. In some embodiments, a system comprisesat least one of foregoing non-transitory computer readable storagemediums, and one or more processors configured to execute theinstructions of the non-transitory computer readable storage medium(s).In some embodiments, a device comprises one or more processorsconfigured to perform any of the steps described above with respect toFIG. 10.

FIG. 11 illustrates an example of a computer in accordance with oneembodiment. Computer 1100 can be a component of a system for simulatingvirtual vehicles on a display according to the systems and methodsdescribed above, such as the devices in physical vehicle 101 orsimulation system 140 described with respect to FIG. 1, or can includethe entire system itself. In some embodiments, computer 1100 isconfigured to execute a method for enhancing a virtual race between aphysical vehicle and a virtual vehicle, such as each of methods 600,700, 800, 900, and 1000 of FIGS. 6, 7, 8, 9, and 10, respectively.

Computer 1100 can be a host computer connected to a network. Computer1100 can be a client computer or a server. As shown in FIG. 11, computer1100 can be any suitable type of microprocessor-based device, such as apersonal computer, workstation, server, videogame console, or handheldcomputing device, such as a phone or tablet. The computer can include,for example, one or more of processor 1110, input device 1120, outputdevice 1130, storage 1140, and communication device 1160. Input device1120 and output device 1130 can generally correspond to those describedabove and can either be connectable or integrated with the computer.

Input device 1120 can be any suitable device that provides input, suchas a touch screen or monitor, keyboard, mouse, or voice-recognitiondevice. Output device 1130 can be any suitable device that providesoutput, such as a touch screen, monitor, printer, disk drive, orspeaker.

Storage 1140 can be any suitable device that provides storage, such asan electrical, magnetic, or optical memory, including a RAM, cache, harddrive, CD-ROM drive, tape drive, or removable storage disk.Communication device 1160 can include any suitable device capable oftransmitting and receiving signals over a network, such as a networkinterface chip or card. The components of the computer can be connectedin any suitable manner, such as via a physical bus or wirelessly.Storage 1140 can be a non-transitory computer-readable storage mediumcomprising one or more programs, which, when executed by one or moreprocessors, such as processor 1110, cause the one or more processors toexecute methods described herein, such as each of methods 600, 700, 800,900, and 1000 of FIGS. 6, 7, 8, 9, and 10, respectively.

Software 1150, which can be stored in storage 1140 and executed byprocessor 1110, can include, for example, the programming that embodiesthe functionality of the present disclosure (e.g., as embodied in thesystems, computers, servers, and/or devices as described above). In someembodiments, software 1150 can be implemented and executed on acombination of servers such as application servers and database servers.

Software 1150, or part thereof, can also be stored and/or transportedwithin any computer-readable storage medium for use by or in connectionwith an instruction execution system, apparatus, or device, such asthose described above, that can fetch and execute instructionsassociated with the software from the instruction execution system,apparatus, or device. In the context of this disclosure, acomputer-readable storage medium can be any medium, such as storage1140, that can contain or store programming for use by or in connectionwith an instruction execution system, apparatus, or device.

Software 1150 can also be propagated within any transport medium for useby or in connection with an instruction execution system, apparatus, ordevice, such as those described above, that can fetch and executeinstructions associated with the software from the instruction executionsystem, apparatus, or device. In the context of this disclosure, atransport medium can be any medium that can communicate, propagate, ortransport programming for use by or in connection with an instructionexecution system, apparatus, or device. The transport-readable mediumcan include, but is not limited to, an electronic, magnetic, optical,electromagnetic, or infrared wired or wireless propagation medium.

Computer 1100 may be connected to a network, which can be any suitabletype of interconnected communication system. The network can implementany suitable communications protocol and can be secured by any suitablesecurity protocol. The network can comprise network links of anysuitable arrangement that can implement the transmission and receptionof network signals, such as wireless network connections, T1 or T3lines, cable networks, DSL, or telephone lines.

Computer 1100 can implement any operating system suitable for operatingon the network. Software 1150 can be written in any suitable programminglanguage, such as C, C++, Java, or Python. In various embodiments,application software embodying the functionality of the presentdisclosure can be deployed in different configurations, such as in aclient/server arrangement or through a Web browser as a Web-basedapplication or Web service, for example.

The foregoing description sets forth exemplary methods, parameters andthe like. It should be recognized, however, that such description is notintended as a limitation on the scope of the present disclosure but isinstead provided as a description of exemplary embodiments. Theillustrative embodiments described above are not intended to beexhaustive or to limit the disclosure to the precise forms disclosed.Many modifications and variations are possible in view of the aboveteachings. The embodiments were chosen and described to best explain theprinciples of the disclosed techniques and their practical applications.Others skilled in the art are thereby enabled to best utilize thetechniques and various embodiments with various modifications as aresuited to the particular use contemplated.

Although the disclosure and examples have been fully described withreference to the accompanying figures, it is to be noted that variouschanges and modifications will become apparent to those skilled in theart. Such changes and modifications are to be understood as beingincluded within the scope of the disclosure and examples as defined bythe claims. In the foregoing description of the disclosure andembodiments, reference is made to the accompanying drawings, in whichare shown, by way of illustration, specific embodiments that can bepracticed. It is to be understood that other embodiments and examplescan be practiced, and changes can be made without departing from thescope of the present disclosure.

Although the foregoing description uses terms first, second, etc. todescribe various elements, these elements should not be limited by theterms. These terms are only used to distinguish one element fromanother. For example, a first virtual vehicle could be termed a secondvirtual vehicle, and, similarly, a second virtual vehicle could betermed a first touch, without departing from the scope of the variousdescribed embodiments.

In addition, it is also to be understood that the singular forms “a,”“an,” and “the” used in the foregoing description are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It is also to be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items. It is further to beunderstood that the terms “includes, “including,” “comprises,” and/or“comprising,” when used herein, specify the presence of stated features,integers, steps, operations, elements, components, and/or units but donot preclude the presence or addition of one or more other features,integers, steps, operations, elements, components, units, and/or groupsthereof.

The term “if” may be construed to mean “when” or “upon” or “in responseto determining” or “in response to detecting,” depending on the context.Similarly, the phrase “if it is determined” or “if [a stated conditionor event] is detected” may be construed to mean “upon determining” or“in response to determining” or “upon detecting [the stated condition orevent]” or “in response to detecting [the stated condition or event],”depending on the context.

In some embodiments, a non-transitory computer readable storage mediumstores one or more programs configured to be executed by one or moreprocessors of an electronic device with a display, the one or moreprograms including instructions for implementing any of the stepsdescribed or claimed herein. The present disclosure also relates to adevice for performing the operations herein. This device may bespecially constructed for the required purposes, or it may include ageneral purpose computer selectively activated or reconfigured by acomputer program stored in the computer. Such a computer program may bestored in a non-transitory, computer readable storage medium, such as,but not limited to, any type of disk, including floppy disks, opticaldisks, CD-ROMs, magnetic-optical disks, read-only memories (ROMs),random access memories (RAMs), EPROMs, EEPROMs, magnetic or opticalcards, application specific integrated circuits (ASICs), or any type ofmedia suitable for storing electronic instructions, and each coupled toa computer system bus. Furthermore, the computers referenced in thisdisclosure may include a single processor or may be architecturesemploying multiple processor designs for increased computing capability.

The methods, devices, and systems described herein are not inherentlyrelated to any particular computer or other apparatus. Variousgeneral-purpose systems may also be used with programs in accordancewith the teachings herein, or it may prove convenient to construct amore specialized apparatus to perform the required method steps. Therequired structure for a variety of these systems will appear from thedescription below. In addition, the present disclosure is not describedwith reference to any particular programming language. It will beappreciated that a variety of programming languages may be used toimplement the teachings of the present disclosure as described herein.

The invention claimed is:
 1. A method for displaying a virtual vehiclecomprising: providing a position of a point of view at a racecourse;calculating a virtual world comprising the virtual vehicle and arepresentation of a physical object at a virtual position, the physicalobject positioned at the racecourse; calculating a virtual position of apoint of view within the virtual world based on the position of thepoint of view at the racecourse; calculating a portion of the virtualvehicle within the virtual world that is visible from the virtualposition of the point of view, wherein the portion of the virtualvehicle visible from the virtual position of the point of view comprisesa portion of the virtual vehicle that is unobscured, from the virtualposition of the point of view, by the representation of the physicalobject at the virtual position of the physical object; and providing, toa display system, the portion of the virtual vehicle visible from thevirtual position of the point of view.
 2. The method of claim 1, whereincalculating a portion of the virtual vehicle within the virtual worldcomprises calculating, at a simulation system, the portion of thevirtual vehicle within the virtual world, and wherein the simulationsystem comprises at least one of a computer system at the racecourse anda computer system remote from the racecourse.
 3. The method of claim 1,wherein providing a position of a point of view at a racecoursecomprises providing respective positions of multiple points of view atthe racecourse, wherein calculating a virtual position of the point ofview within the virtual world comprises calculating respective virtualpositions of the points of view within the virtual world based on therespective positions of the points of view at the racecourse, whereincalculating a portion of the virtual vehicle within the virtual worldthat is visible from the virtual position of the point of view comprisescalculating respective portions of the virtual vehicle within thevirtual world that are visible from the corresponding virtual positionsof the points of view, wherein the respective portions of the virtualvehicle within the virtual world that are visible from the correspondingvirtual positions of the points of view comprise respective portions ofthe virtual vehicle that are unobscured, from the respective virtualposition, by the representation of the physical object, and whereinproviding, to the display system, the portion of the virtual vehiclevisible from the virtual position of the point of view comprisesproviding the respective portions of the virtual vehicle visible fromthe virtual positions of the points of view.
 4. The method of claim 3,wherein the display system comprises multiple display screens eachdisplaying a respective portion of the virtual vehicle.
 5. The method ofclaim 3, wherein the physical object comprises separate physical objectsfor each point of view.
 6. The method of claim 1, wherein the physicalobject is a physical vehicle on the racecourse.
 7. The method of claim1, further comprising: generating, at the display system, arepresentation of the portion of the virtual vehicle visible from thevirtual position of the point of view; and displaying a series ofrepresentations of the virtual vehicle over a period of time to simulatea trajectory of the virtual vehicle on the racecourse, wherein theseries of representations comprises the representation of the portion ofthe virtual vehicle visible from the virtual position of the point ofview.
 8. The method of claim 1, wherein providing a position of a pointof view comprises providing at least one of a spatial position, a gazedirection, or a focus point.
 9. The method of claim 1, wherein the pointof view comprises a point of view of an operator of a physical vehicle,a point of view of an audience member present at the racecourse andobserving the physical object, or a point of view of a camera present atthe racecourse and imaging the physical object.
 10. The method of claim1, wherein calculating a portion of the virtual vehicle within thevirtual world that is visible from the virtual position of the point ofview comprises calculating a field of view from the virtual position ofthe point of view and wherein providing, to the display system, theportion of the virtual vehicle visible from the virtual position of thepoint of view comprises providing the portions of the virtual vehiclewithin the field of view.
 11. The method of claim 1, wherein calculatingthe virtual world comprises transforming physical coordinates of thephysical object to virtual coordinates in the virtual world and whereina virtual position of the physical object in the virtual world comprisesthe virtual coordinates.
 12. A non-transitory computer readable storagemedium storing a program configured to be executed by a processor of anelectronic device, the program including instructions for implementingthe following steps: receiving a position of a point of view at aracecourse from a sensor; calculating a virtual world comprising avirtual vehicle and a representation of a physical object at a virtualposition; calculating a virtual position of the point of view within thevirtual world based on the position of the point of view at theracecourse; calculating a portion of the virtual vehicle within thevirtual world that is visible from the virtual position of the point ofview, wherein the portion of the virtual vehicle visible from thevirtual position of the point of view comprises a portion of the virtualvehicle that is unobscured, from the virtual position of the point ofview, by the representation of the physical object at the virtualposition of the physical object; and outputting, to a display system,the portion of the virtual vehicle visible from the virtual position ofthe point of view.
 13. The non-transitory computer readable storagemedium of claim 12, wherein calculating a portion of the virtual vehiclewithin the virtual world comprises calculating, at a simulation system,the portion of the virtual vehicle within the virtual world, and whereinthe simulation system comprises at least one of a computer system at theracecourse and a computer system remote from the racecourse.
 14. Thenon-transitory computer readable storage medium of claim 12, whereinproviding a position of a point of view at a racecourse comprisesproviding respective positions of multiple points of view at theracecourse, wherein calculating a virtual position of the point of viewwithin the virtual world comprises calculating respective virtualpositions of the points of view within the virtual world based on therespective positions of the points of view at the racecourse, whereincalculating a portion of the virtual vehicle within the virtual worldthat is visible from the virtual position of the point of view comprisescalculating respective portions of the virtual vehicle within thevirtual world that are visible from the corresponding virtual positionsof the points of view, wherein the respective portions of the virtualvehicle within the virtual world that are visible from the correspondingvirtual positions of the points of view comprise respective portions ofthe virtual vehicle that are unobscured, from the respective virtualposition, by the representation of the physical object, and whereinproviding, to the display system, the portion of the virtual vehiclevisible from the virtual position of the point of view comprisesproviding the respective portions of the virtual vehicle visible fromthe virtual positions of the points of view.
 15. The non-transitorycomputer readable storage medium of claim 14, wherein the display systemcomprises multiple display screens each displaying a respective portionof the virtual vehicle.
 16. The non-transitory computer readable storagemedium of claim 14, wherein the physical object comprises separatephysical objects for each point of view.
 17. The non-transitory computerreadable storage medium of claim 12, wherein the physical object is aphysical vehicle on the racecourse.
 18. The non-transitory computerreadable storage medium of claim 12, wherein the program includesinstructions for implementing the following steps: generating, at thedisplay system, a representation of the portion of the virtual vehiclevisible from the virtual position of the point of view; and displaying aseries of representations of the virtual vehicle over a period of timeto simulate a trajectory of the virtual vehicle on the racecourse,wherein the series of representations comprises the representation ofthe portion of the virtual vehicle visible from the virtual position ofthe point of view.
 19. The non-transitory computer readable storagemedium of claim 12, wherein providing a position of a point of viewcomprises providing at least one of a spatial position, a gazedirection, or a focus point.
 20. The non-transitory computer readablestorage medium of claim 12, wherein the point of view comprises a pointof view of an operator of a physical vehicle, a point of view of anaudience member present at the racecourse and observing the physicalobject, or a point of view of a camera present at the racecourse andimaging the physical object.
 21. The non-transitory computer readablestorage medium of claim 12, wherein calculating a portion of the virtualvehicle within the virtual world that is visible from the virtualposition of the point of view comprises calculating a field of view fromthe virtual position of the point of view and wherein providing, to thedisplay system, the portion of the virtual vehicle visible from thevirtual position of the point of view comprises providing the portionsof the virtual vehicle within the field of view.
 22. The non-transitorycomputer readable storage medium of claim 12, wherein calculating thevirtual world comprises transforming physical coordinates of thephysical object to virtual coordinates in the virtual world and whereina virtual position of the physical object in the virtual world comprisesthe virtual coordinates.